I did a demo this evening and, for the first time, out of the sealed box comes a refurbished device with v1.0.0 vendor firmware on it. I use the v1.1.3 unsigned installer firmware, it reboots, I try to sysupgrade from the recovery and things are haywire: reboots into the recovery as default. From a u-boot console, I can reboot to the production firmware, but if I sysupgrade to openwrt-24.10.0-rc4-mediatek-mt7622-linksys_e8450-squashfs-sysupgrade.bin, it kernel panics during boot. If the production firmware is a snapshot from circa August 23, 2024, I can select it from the u-boot menu, and it boots to something that looks normal, with the rootfs_data partition mounted. Another oddity, is that warm reboots fail, hanging at:
root@OpenWrt:~# reboot
root@OpenWrt:~# [ 81.631862] mt7530-mdio mdio-bus:1f lan1: left allmulticast mode
[ 81.637898] mt7530-mdio mdio-bus:1f lan1: left promiscuous mode
[ 81.644093] br-lan: port 1(lan1) entered disabled state
[ 81.708414] mt7530-mdio mdio-bus:1f lan2: left allmulticast mode
[ 81.714552] mt7530-mdio mdio-bus:1f lan2: left promiscuous mode
[ 81.720685] br-lan: port 2(lan2) entered disabled state
[ 81.768128] mt7530-mdio mdio-bus:1f lan3: left allmulticast mode
[ 81.774252] mt7530-mdio mdio-bus:1f lan3: left promiscuous mode
[ 81.780393] br-lan: port 3(lan3) entered disabled state
[ 81.837203] mt7530-mdio mdio-bus:1f lan4: left allmulticast mode
[ 81.843321] mtk_soc_eth 1b100000.ethernet eth0: left allmulticast mode
[ 81.849888] mt7530-mdio mdio-bus:1f lan4: left promiscuous mode
[ 81.855911] br-lan: port 4(lan4) entered disabled state
[ 82.366265] mt7530-mdio mdio-bus:1f wan: Link is Down
[ 82.373904] mtk_soc_eth 1b100000.ethernet eth0: Link is Down
[ 86.439990] reboot: Restarting system
F0: 102B 0000
F6: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0400 0041 [0000]
G0: 1190 0000
T0: 0000 02F1 [000F]
Jump to BL
NOTICE: BL2: v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
NOTICE: BL2: Built : 22:48:06, Aug 12 2024
NOTICE: WDT: [40000000] Software reset (reboot)
NOTICE: CPU: MT7622
Power cycle brings me back to a u-boot menu.
Looking closer at the serial console logs, the 24.10.0-rc4 is hanging at not finding a bunch of /dev/mtdblockN's, so it looks like it's a non-UBI version (which I thought didn't exist anymore). So, that starts to make sense.
EDIT: Oh, lol, I was blind. Squinting harder, there IS a ubi version. I somehow got the impression that there weren't any non-UBI versions anymore. Can't read and talk at the same time.
EDIT2: /dev/mtdblock0 (bl2), /dev/ubi0_0 (fip), /dev/ubi0_3 (ubootenv2) and /dev/ubi0_4 (recovery) sha256sums are the same as on a working device. Now reboot is rebooting into the 24.10.0-rc4 rather than recovery, so that's good. But I'm still seeing the hang, quoted above, during a warm reboot.
A different device running the same firmware build does not hang at that spot, but continues on, thusly:
NOTICE: BL2: v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
NOTICE: BL2: Built : 22:48:06, Aug 12 2024
NOTICE: WDT: [40000000] Software reset (reboot)
NOTICE: CPU: MT7622
NOTICE: SPI-NAND: adjusting SPI-NAND pin drive strength to 12mA
NOTICE: SPI-NAND: W25N01GV (128MB)
NOTICE: UBI: scanning [0x80000 - 0x8000000] ...
NOTICE: UBI: scanning is finished
NOTICE: UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
NOTICE: UBI: VID header offset: 2048 (aligned 2048), data offset: 4096
NOTICE: UBI: Volume fip (Id #0) size is 1015884 bytes
NOTICE: BL2: Booting BL31
NOTICE: BL31: v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
NOTICE: BL31: Built : 22:48:06, Aug 12 2024
U-Boot 2024.07-OpenWrt-r27137-f51cb74473 (Aug 12 2024 - 22:48:06 +0000)
CPU: MediaTek MT7622
Model: mt7622-linksys-e8450-ubi
DRAM: 512 MiB
Core: 49 devices, 21 uclasses, devicetree: separate
MMC:
Loading Environment from UBI... SPI-NAND: W25N01GV (128MB)
Read 126976 bytes from volume ubootenv to 000000005f7bf0c0
Read 126976 bytes from volume ubootenv2 to 000000005f7de100
[...]
What might cause that hang?
Here's a diff of the cold-boot boot logs (with timestamps trimmed) between the problematic device and the happily warm booting device:
--- /tmp/screenlog.2.txt 2025-01-03 04:11:14.347589979 -0800
+++ /tmp/screenlog.9.txt 2025-01-03 04:11:25.488154784 -0800
@@ -1,11 +1,11 @@
F0: 102B 0000
F6: 0000 0000
V0: 0000 0000 [0001]
00: 0000 0000
BP: 0400 0041 [0000]
G0: 1190 0000
-T0: 0000 02EE [000F]
+T0: 0000 02EF [000F]
Jump to BL
NOTICE: BL2: v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
@@ -14,7 +14,7 @@
NOTICE: CPU: MT7622
NOTICE: WDT: disabled
NOTICE: SPI-NAND: adjusting SPI-NAND pin drive strength to 12mA
-NOTICE: SPI-NAND: FM35Q1GA (128MB)
+NOTICE: SPI-NAND: W25N01GV (128MB)
NOTICE: UBI: scanning [0x80000 - 0x8000000] ...
NOTICE: UBI: scanning is finished
NOTICE: UBI: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
@@ -32,7 +32,7 @@
DRAM: 512 MiB
Core: 49 devices, 21 uclasses, devicetree: separate
MMC:
-Loading Environment from UBI... SPI-NAND: FM35Q1GA (128MB)
+Loading Environment from UBI... SPI-NAND: W25N01GV (128MB)
Read 126976 bytes from volume ubootenv to 000000005f7bf0c0
Read 126976 bytes from volume ubootenv2 to 000000005f7de100
OK
@@ -265,7 +265,7 @@
mtk_rng 1020f000.rng: registered RNG driver
loop: module loaded
mtk-ecc 1100e000.ecc: probed
- spi-nand spi2.0: Fidelix SPI NAND was found.
+ spi-nand spi2.0: Winbond SPI NAND was found.
spi-nand spi2.0: 128 MiB, block size: 128 KiB, page size: 2048, OOB size: 64
mtk-snand 1100d000.spi: ECC strength: 4 bits per 512 bytes
2 fixed-partitions partitions found on MTD device spi2.0
@@ -284,13 +284,13 @@
ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
ubi0: good PEBs: 1020, bad PEBs: 0, corrupted PEBs: 0
ubi0: user volume: 8, internal volumes: 1, max. volumes count: 128
- ubi0: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 147074388
+ ubi0: max/mean erase counter: 7/4, WL threshold: 4096, image sequence number: 405897712
ubi0: available PEBs: 0, total reserved PEBs: 1020, PEBs reserved for bad PEB handling: 20
ubi0: background thread "ubi_bgt0d" started, PID 228
OF: Bad cell count for /spi@1100d000/flash@0/partitions
OF: Bad cell count for /spi@1100d000/flash@0/partitions
block ubiblock0_5: created from ubi0:5(fit)
- mtk_soc_eth 1b100000.ethernet eth0: mediatek frame engine at 0xffffffc081300000, irq 126
+ mtk_soc_eth 1b100000.ethernet eth0: mediatek frame engine at 0xffffffc081440000, irq 126
i2c_dev: i2c /dev entries driver
mtk-wdt 10212000.watchdog: Watchdog enabled (timeout=31 sec, nowayout=0)
NET: Registered PF_INET6 protocol family
@@ -416,14 +416,14 @@
Press the [f] key and hit [enter] to enter failsafe mode
Press the [1], [2], [3] or [4] key and hit [enter] to select the debug level
UBIFS (ubi0:7): Mounting in unauthenticated mode
- UBIFS (ubi0:7): background thread "ubifs_bgt0_7" started, PID 732
+ UBIFS (ubi0:7): background thread "ubifs_bgt0_7" started, PID 729
UBIFS (ubi0:7): recovery needed
UBIFS (ubi0:7): recovery completed
UBIFS (ubi0:7): UBIFS: mounted UBI device 0, volume 7, name "rootfs_data"
UBIFS (ubi0:7): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
UBIFS (ubi0:7): FS size: 94724096 bytes (90 MiB, 746 LEBs), max 757 LEBs, journal size 4698112 bytes (4 MiB, 37 LEBs)
UBIFS (ubi0:7): reserved for root: 4474052 bytes (4369 KiB)
- UBIFS (ubi0:7): media format: w5/r0 (latest is w5/r0), UUID B0E57F54-C5CC-4B4E-BA98-185D6690CF85, small LPT model
+ UBIFS (ubi0:7): media format: w5/r0 (latest is w5/r0), UUID 9C96635F-40FE-471D-8AB2-EB103C904C44, small LPT model
mount_root: switching to ubifs overlay
overlayfs: null uuid detected in lower fs '/', falling back to xino=off,index=off,nfs_export=off.
urandom-seed: Seeding with /etc/urandom.seed
@@ -440,8 +440,8 @@
ieee80211 phy0: Selected rate control algorithm 'minstrel_ht'
mt7915e 0000:01:00.0: assign IRQ: got 130
mt7915e 0000:01:00.0: enabling device (0000 -> 0002)
- urngd: v1.0.2 started.
mt7915e 0000:01:00.0: enabling bus mastering
+ urngd: v1.0.2 started.
mt7622-wmac 18000000.wmac: HW/SW Version: 0x8a108a10, Build Time: 20190801210006a
mtk-pcie 1a143000.pcie: msi#0 address_hi 0x0 address_lo 0x44fb50c0
I'm not seeing anything that looks significant, other than (maybe?) the flash vendor, but I have other devices with Fidelix without apparent problems.