Firmware flashing broken since r30754 (sysupgrade from snapshot r36702..r36762 may fail)

ghoffman · October 3, 2026, 11:11am

i've got mx4300 as AP. since r36754, i cant flash new builds using owut, sysupgrade or the graphical interface. i've tried a default build form firmware-selector and that also will not flash. i'm my device:

Hostname	ap253
Model	Linksys MX4300
Architecture	ARMv8 Processor rev 4
Target Platform	qualcommax/ipq807x
Firmware Version	OpenWrt SNAPSHOT r36754-c1b3943aa3 / LuCI Master 26.273.63965~32775cf
Kernel Version	6.18.54

wondering of rthe the recent changes to 'mtd' have created a problem. anyone else seeing this?

frollic · October 3, 2026, 11:15am

Use stable?

ghoffman · October 3, 2026, 11:47am

the problem that i'm reporting is that flashing 'from' r36754 fails. i've tried flashing 25.12.5 but i'. stuck back on r36754.
fw_setenv boot_part 2 reboots to partttion 1. so i'm stuck on it.

seems like a mtd / mtd tools problem in addtion to things reported in issues

frollic · October 3, 2026, 1:53pm

Assuming you can't figure out what's wrong via ssh, you'll need serial to investigate/report the issue and be able to get out of the snapshot you're currently on.

You could try to add an automatic TFTP or USB recovery sequence to U-Boot prior to booting from flash, from OpenWRT, but if you get it wrong, you'll def need serial to recover.

ghoffman · October 3, 2026, 4:10pm

its something with recent shapshots on kernel 6.18.54; a 'later' build with 6.18.52 will take a sysupgrade. i might have had a frankenbuild from the owut server. for now, i have reverted to 25.12.5.

i have 3 of these devices all as AP's, and only bricked one of them. my route was to use mtd write from the openwrt cli, going back to oem firmware on both partitions and then restarting the openwrt insertion process.

ghoffman · October 4, 2026, 11:59am

fyi - this issue has afflicted all my devices that had r36754, including mx8500, whw01, and rt4330w. i've gone back to 25.12.5 on all these devices

how should i go about reporting this issue? i cold not find any other reports, so maybe it was a temporary imagebuilder issue that i was lucky enough to catch.

thanks for your frequent and consistent help in these forums

frollic · October 4, 2026, 12:04pm

You need to get serial, an issue stating "it's not working" won't get picked up, logs are most likely the 1st thing they'll ask for anyway.

The reporting is done on the page you got earlier.

hnyman · October 5, 2026, 3:23pm

I just compiled a new firmware for qualcommax/ipq807x/DL-WRX36 and it fails to boot.
My previous firmware was r36743-c759267c92 compiled at 20260930T2056. And I am now trying sysupgrade from it.

Likely in the similar error as you @ghoffman

I have serial there, and it seems to hang at waiting rootfs:

[    1.691611] 0x000001000000-0x000007100000 : "rootfs"
[    1.766955] mtd: setting mtd18 (rootfs) as root device
[    1.767246] mtdsplit: no squashfs found in "rootfs"
[    1.771051] 0x000007100000-0x000007a00000 : "0:wififw"
[    1.782869] 0x000007a00000-0x00000db00000 : "rootfs_1"
[    1.854767] 0x00000db00000-0x00000e400000 : "0:wififw_1"
[    1.861946] 0x00000e400000-0x00000fa00000 : "ubifs"
[    1.878612] 0x00000fa00000-0x00000fa80000 : "0:ethphyfw"
[    1.879480] 0x00000fa80000-0x00000fd00000 : "certificate"
[    2.237295] cpufreq: cpufreq_policy_online: CPU0: Running at unlisted initial frequency: 800000 kHz, changing to: 1017600 kHz
[    2.239236] remoteproc remoteproc0: q6wcss is available
[    2.578119] qca-ppe 3a000000.ppe: configuring for fixed/internal link mode
[    2.578181] qca-ppe 3a000000.ppe: Link is Up - 1Gbps/Full - flow control off
[    2.758628] qca-ppe 3a000000.ppe lan4 (uninitialized): PHY [90000.mdio-1:00] driver [Qualcomm QCA8075] (irq=POLL)
...
[    3.040679] qca-edma 3ab00000.edma eth0: entered promiscuous mode
[    3.049121] DSA: tree 0 setup
[    3.055341] ubi0: attaching mtd18
[    3.485058] ubi0: scanning is finished
[    3.528272] ubi0: attached mtd18 (name "rootfs", size 97 MiB)
[    3.528310] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    3.533003] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    3.539796] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    3.546633] ubi0: good PEBs: 776, bad PEBs: 0, corrupted PEBs: 0
[    3.553410] ubi0: user volume: 4, internal volumes: 1, max. volumes count: 128
[    3.559664] ubi0: max/mean erase counter: 177/119, WL threshold: 4096, image sequence number: 634945484
[    3.566690] ubi0: available PEBs: 0, total reserved PEBs: 776, PEBs reserved for bad PEB handling: 40
[    3.576003] ubi0: background thread "ubi_bgt0d" started, PID 724
[    3.576716] block ubiblock0_2: created from ubi0:2(rootfs)
[    3.591554] clk: Disabling unused[    3.600982] check access for rdinit=/init failed: -2, ignoring
[    3.601087] Waiting for root device /dev/ubiblock0_1...
[   13.927805] qcom,gcc-ipq8074 1800000.clock-controller: sync_state() pending due to 8a00000.usb
[   33.767788] l11: disabling

In my archive from March 2026, a successful boot goes like this:

[    2.023598] ubi0: background thread "ubi_bgt0d" started, PID 668
[    2.033457] block ubiblock0_1: created from ubi0:1(rootfs)
[    2.039143] clk: Disabling unused clocks
[    2.044702] PM: genpd: Disabling unused power domains
[    2.051414] VFS: Mounted root (squashfs filesystem) readonly on device 254:0.
[    2.051737] Freeing unused kernel memory: 960K
[    2.057641] Run /sbin/init as init process

First guess is that the crucial difference is
[ 3.576716] block ubiblock0_2: created from ubi0:2(rootfs)
vs
[ 2.033457] block ubiblock0_1: created from ubi0:1(rootfs)
breaking the rootfs finding.

That makes me to think that it may be related to this merged by @nbd

github.com/openwrt/openwrt

base-files: keep kernel and rootfs UBI volume IDs with provisioning

committed 08:29PM - 01 Oct 26 UTC

c127dev

+4 -2

Since sysupgrade creates the provisioning volume by default, it takes the lowest… free volume ID before the kernel and rootfs volumes are created. This shifts their IDs by one, so devices that hardcode the rootfs volume in root= (e.g. root=/dev/ubiblock0_1) no longer boot after a sysupgrade. Create the provisioning volume with the last volume ID instead. A provisioning volume that already exists at a low ID is kept and has to be removed by hand. Fixes: 725a1dc534a1 ("base-files: create the provisioning partition on sysupgrade by default") Fixes: https://github.com/openwrt/openwrt/issues/25501 Signed-off-by: Johan Alvarado <contact@c127.dev>

It explicitly says that:

Since sysupgrade creates the provisioning volume by default, it takes the
lowest free volume ID before the kernel and rootfs volumes are created.
This shifts their IDs by one, so devices that hardcode the rootfs volume
in root= (e.g. root=/dev/ubiblock0_1) no longer boot after a sysupgrade.

EDIT:
that previously mentioned commit is apparently the fix, not the problem itself, which was from two days earlier:

LS3434 · October 5, 2026, 3:36pm

So devs are merging stuff, without testing, that bricks routers ? That's nice...

hnyman · October 5, 2026, 3:38pm

... bricking just some devices, not all...

(there is no real possibility to test all devices for all changes)

LS3434 · October 5, 2026, 3:40pm

Well, don't most devices in qualcommax target have rootfs volumes hardcoded ? If so it would be nice to check first.

hnyman · October 5, 2026, 3:46pm

Yeah,
just checked from DTS, and it is hardcoded for DL-WRX36 :

github.com/openwrt/openwrt

target/linux/qualcommax/dts/ipq8072-dl-wrx36.dts

29cf0022e


      
          	aliases {
          		led-boot = &led_system_red;
          		led-failsafe = &led_system_red;
          		led-running = &led_system_blue;
          		led-upgrade = &led_system_red;
          		serial0 = &blsp1_uart5;
          	};
          
          	chosen {
          		stdout-path = "serial0:115200n8";
          		bootargs-append = " root=/dev/ubiblock0_1";
          	};
          
          	keys {
          		compatible = "gpio-keys";
          
          		reset {
          			label = "reset";
          			gpios = <&tlmm 34 GPIO_ACTIVE_LOW>;
          			linux,code = <KEY_RESTART>;
          		};

LS3434 · October 5, 2026, 3:50pm

Just checked, and someone reported that ipq50xx is affected too haha So, pretty much most of qualcommax target...

hnyman · October 5, 2026, 4:09pm

I guess that that problem only concerns if sysupgrading from a build

AFTER the r36702-725a1dc534 "create provisioning partition by default" on 29 September and
BEFORE the later fix r36763-3f26ab3d4d "keep kernel and rootfs UBI volume IDs with provisioning" merged on 1 October.

So, both my own build r36743 and ghoffman's r30754 just fall into the problematic time period.

Just need to figure the best way to escape the trap.
(I have the DL-WRX36 fallback USB stick bootcmd installed, so that will likely offer a way for me.)

EDIT:
related bug:

github.com/openwrt/openwrt

Creation of provisioning partition (now default for sysupgrade) bricks device(s)

opened 10:08AM - 30 Sep 26 UTC

closed 08:30PM - 01 Oct 26 UTC

mirko

target/ipq806x bug SNAPSHOT Self Built Image Supported Device

### Describe the bug sysupgrade now creating provisioning partitions by default… (725a1dc534a1ad1008ae0412925100fef66a776f) appears to at least brick the Aruba AP-325 device. It expects the rootfs-part to be the first volume (index 0), which however the provisioning part becomes after an sysupgrade with (by default) prov-part creation enabled. The image boots with: `ubi.mtd=aos1 ubi.mtd=ubifs ubi.block=0,aos1 root=/dev/ubiblock0_0` Result is: ``` [ 1.808048] ubi0: attaching mtd1 [ 1.964462] random: crng init done [ 2.311398] ubi0: scanning is finished [ 2.365050] ubi0: attached mtd1 (name "aos1", size 32 MiB) [ 2.365083] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes [ 2.369432] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048 [ 2.376387] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096 [ 2.383147] ubi0: good PEBs: 256, bad PEBs: 0, corrupted PEBs: 0 [ 2.389993] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128 [ 2.396243] ubi0: max/mean erase counter: 4/2, WL threshold: 4096, image sequence number: 0 [ 2.403200] ubi0: available PEBs: 175, total reserved PEBs: 81, PEBs reserved for bad PEB handling: 20 [ 2.411606] ubi0: background thread "ubi_bgt0d" started, PID 66 [ 2.412022] block ubiblock0_1: created from ubi0:1(aos1) [ 2.426734] ubi1: attaching mtd2 [ 3.444040] ubi1: scanning is finished [ 3.464002] ubi1: attached mtd2 (name "ubifs", size 64 MiB) [ 3.464035] ubi1: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes [ 3.468472] ubi1: min./max. I/O unit sizes: 2048/2048, sub-page size 2048 [ 3.475398] ubi1: VID header offset: 2048 (aligned 2048), data offset: 4096 [ 3.482185] ubi1: good PEBs: 511, bad PEBs: 1, corrupted PEBs: 0 [ 3.489027] ubi1: user volume: 2, internal volumes: 1, max. volumes count: 128 [ 3.495264] ubi1: max/mean erase counter: 6/1, WL threshold: 4096, image sequence number: 0 [ 3.502239] ubi1: available PEBs: 0, total reserved PEBs: 511, PEBs reserved for bad PEB handling: 19 [ 3.510625] ubi1: background thread "ubi_bgt1d" started,�[ 3.526256] /dev/root: Can't open blockdev [ 3.526296] VFS: Cannot open root device "/dev/ubiblock0_0" or unknown-block(0,0): error -6 [ 3.529248] Please append a correct "root=" boot option; here are the available partitions: [ 3.537576] 1f00 32768 mtdblock0 [ 3.537586] (driver?) [ 3.549990] 1f01 32768 mtdblock1 [ 3.549999] (driver?) [ 3.556562] 1f02 65536 mtdblock2 [ 3.556570] (driver?) [ 3.563008] 1f03 128 mtdblock3 [ 3.563016] (driver?) [ 3.569521] 1f04 128 mtdblock4 [ 3.569530] (driver?) [ 3.576087] 1f05 256 mtdblock5 [ 3.576095] (driver?) [ 3.582541] 1f06 512 mtdblock6 [ 3.582549] (driver?) [ 3.589052] 1f07 64 mtdblock7 [ 3.589061] (driver?) [ 3.595643] 1f08 64 mtdblock8 [ 3.595651] (driver?) [ 3.602072] 1f09 512 mtdblock9 [ 3.602080] (driver?) [ 3.608584] 1f0a 512 mtdblock10 [ 3.608592] (driver?) [ 3.615511] 1f0b 1024 mtdblock11 [ 3.615520] (driver?) [ 3.622037] 1f0c 64 mtdblock12 [ 3.622045] (driver?) [ 3.628702] 1f0d 256 mtdblock13 [ 3.628711] (driver?) [ 3.635287] 1f0e 64 mtdblock14 [ 3.635295] (driver?) [ 3.641828] 1f0f 64 mtdblock15 [ 3.641836] (driver?) [ 3.648501] 1f10 64 mtdblock16 [ 3.648509] (driver?) [ 3.655092] 1f11 64 mtdblock17 [ 3.655101] (driver?) [ 3.661620] 1f12 320 mtdblock18 [ 3.661628] (driver?) [ 3.668276] fe00 6820 ubiblock0_1 [ 3.668284] (driver?) [ 3.674866] List of all bdev filesystems: [ 3.677071] squashfs [ 3.677077] [ 3.683405] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0 ``` ### OpenWrt version bb5ae5fd5acc10f3822017681941eed07fb214a3 ### OpenWrt release SNAPSHOT ### OpenWrt target/subtarget ipq806x/generic ### Device Aruba AP-32x ### Image kind Self-built image ### Steps to reproduce _No response_ ### Actual behaviour _No response_ ### Expected behaviour _No response_ ### Additional info _No response_ ### Diffconfig ```text ``` ### Terms - [x] I am reporting an issue for OpenWrt, not an unsupported fork.

EDIT2:

I fixed my own things by:

boot DL-WRX36 with the USB stick recovery boot to an old OpenWrt version uimage.
compile a personal new firmware image with DTS patched with root=/dev/ubiblock0_2 so that it will boot with the current ubi contents.
Transfer the image to router /tmp with scp. Then sysupgrade to that image
manually delete the "provisioning" ubi0_0 volume with ubirmvol
sysupgrade from ssh console to a new firmware with normal root=/dev/ubiblock0_1 with sysupgrade --no-provisioning <firmware_name>
Then sysupgrade normally to the same normal image.

Now I have the provisioning volume there at ubi 0_127, and kernel is normal ubi0_0 and rootfs ubi0_1:

root@router5:~# ubinfo -a
UBI version:                    1
Count of UBI devices:           1
UBI control device major/minor: 10:256
Present UBI devices:            ubi0

ubi0
Volumes count:                           4
Logical eraseblock size:                 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks:     776 (98533376 bytes, 93.9 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  40
Current maximum erase counter value:     180
Minimum input/output unit size:          2048 bytes
Character device major/minor:            246:0
Present volumes:                         0, 1, 2, 127

Volume ID:   0 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        47 LEBs (5967872 bytes, 5.6 MiB)
State:       OK
Name:        kernel
Character device major/minor: 246:1
-----------------------------------
Volume ID:   1 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        151 LEBs (19173376 bytes, 18.2 MiB)
State:       OK
Name:        rootfs
Character device major/minor: 246:2
-----------------------------------
Volume ID:   2 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        532 LEBs (67551232 bytes, 64.4 MiB)
State:       OK
Name:        rootfs_data
Character device major/minor: 246:3
-----------------------------------
-----------------------------------
...
-----------------------------------
Volume ID:   127 (on ubi0)
Type:        dynamic
Alignment:   1
Size:        2 LEBs (253952 bytes, 248.0 KiB)
State:       OK
Name:        provisioning
Character device major/minor: 246:128
root@router5:~#

c127dev · October 5, 2026, 7:32pm

If you're running a snapshot r36702..r36762, upgrade with:

sysupgrade --no-provisioning <image>

Run it from the shell. LuCI and owut don't pass the flag.

The flashing is done by the running build's sysupgrade script, so this applies to any target image, including 25.12.x.

Without the flag, that script creates the provisioning vol at ID 0 and the new image hangs at Waiting for root device /dev/ubiblock0_1. A board that already hangs needs serial, web recovery or USB recovery first.

With a fixed build (r36763+), the next normal sysupgrade puts provisioning at the last volume ID.

Tested on a Mercusys MR80X v2.

hnyman · October 5, 2026, 7:49pm

Yeah, the sad part is that if you are now running a faulty build, you will likely notice this discussion only after a failure with normal sysupgrade, and then you are already in trouble.