Yes, it's not as super safe as current snapshots which also do scrubbing now, but it's at least as safe as stock firmware and not endangered to become a brick due to single bitflips.
I suppose there is some benefit in mass and tested migration using the same installer version. What I mean is if lots of users all use the same installer version and report success, then this gives some added peace of mind for the safety of the install. Whereas the more users in different boats, we don't know which boats to trust.
Since upgrade from v2.4 to v2.9 TF-A caused big headaches, and couldn't v2.10 or v2.11+ also contain further issues? Or is this unlikely? Unlikely to be as bad as the OKD issue, in any event?
1 Like
What about those already on snapshot (without suffering from OKD yet)?
What's the proper way to fix ? (without serial please)
1 Like
I think it's described above in a few places already. Like here if I am not mistaken.
1 Like
It would be essentially the same as what I posted, but you'd use the newer BL2 image.
1 Like
Regarding the snapshot, is anyone running the snapshot successfully without big issues ?
For a lot of users this whole boot-thing-issue is quite confusing.
The question is: which update-paths are safe and which aren´t?
1 Like
Hi,
I am running Feb snapshots on a non-OKD router. I have the boot backup from before moving to FIP-on-UBI . How can I do a boot backup after I have FIP-on-UBI? Again as Daniel mentions in github mkdir /tmp/boot_backup mount -t ubifs ubi0:boot_backup /tmp/boot_backup or something else? In the end do I really need a backup as I already have pre moving FiP to UBI - will the recovery using mtk_uartboot tool be easy?
Thanks
Kr
K
For those that follow all the technical specifics and instructions in this thread without any difficulty, perhaps reading the below might offer something of a simulation of what it might be like for the remaining individuals:
You just flip the reverse rollover UBI v2.4 to v2.10 or v2.9 to v2.10 and inspect the individual HEX values of the blob thereby to achieve a synergistic bitflip binary that you can flash using a serial device in which you solder the TX, PX, RQ and RL lines having verified with a multimeter that the voltages are between 0.3 and 0.4, and issue the sequence
boomshakalakaover the wire encoded with z-type encryption to bypass the BCH engine and then close your eyes and reboot. Then all should be OK.
I am reminded of this scene from the series 'Maid'.
I think the challenge really is that documenting stuff clearly is boring and time consuming, and all of this is voluntary and hence one is relying in part on the good nature and generosity of others. Also OpenWrt seems to me to be made by nerds for nerds, and nerds aren't normally that great at documenting things anyway.
Normally things become clear to me when I keep an eye on this thread and parse about six different explanations of the same thing.
2 Likes
I wonder what I should do if I am on a snapshot and I know that my next boot will OKD (the last time I rebooted I had to use the freezer to get it to boot, so I think my next reboot is OKD for the 4th time).
I'm wondering if perhaps I should follow these steps to rewrite my FIP, then follow the quoted steps? Or is there a more straightforward way?
1 Like
From "do from SSH console" perspective, you have likely found the easiest path
- First rewrite FIP in UBI via ubiupdatevol
- Sysupgrade to new snapshot to get correct kmod-mtd-rw
- Use mtd to rewrite bl2 (preloader.bin)
You quoted & linked my messages above, and reflecting those: I have used ubiupdatevol twice to rewrite fip, and I used the above referenced process to get the new bl2 into the router. It is pretty straightforward when you already have "fip in UBI" snapshot in use.
2 Likes
Just to report, that OKD-fix was applied correctly (or at least I didn't brick the router:)) to a router running Feb Snapshot. All went fine using @hnyman instructions. I did a backup of the boot from luci directly. I used attended sysupgrade and added kmod-mtd-rw before I requested a build.
Once again the steps taken from @hnyman
root@Belkin3:~# cd /tmp
root@Belkin3:/tmp# insmod mtd-rw.ko i_want_a_brick=1
root@Belkin3:/tmp# mtd -p 0x0 write /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin /dev/mtd0
Couldn't open image file: /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin!
root@Belkin3:/tmp# mtd -p 0x0 write /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin /dev/mtd0
Unlocking /dev/mtd0 ...
Writing from /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin to /dev/mtd0 ...
root@Belkin3:/tmp# mtd -p 0x20000 write /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin /dev/mtd0
Unlocking /dev/mtd0 ...
Seeking on mtd device '/dev/mtd0' to: 131072
Writing from /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin to /dev/mtd0 ...
root@Belkin3:/tmp# mtd -p 0x40000 write /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin /dev/mtd0
Unlocking /dev/mtd0 ...
Seeking on mtd device '/dev/mtd0' to: 262144
Writing from /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin to /dev/mtd0 ...
root@Belkin3:/tmp# mtd -p 0x60000 write /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin /dev/mtd0
Unlocking /dev/mtd0 ...
Seeking on mtd device '/dev/mtd0' to: 393216
Writing from /tmp/openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin to /dev/mtd0 ...
root@Belkin3:/tmp# sync
root@Belkin3:/tmp# /dev/mtd0ro
-ash: /dev/mtd0ro: Permission denied
root@Belkin3:/tmp# grep "(release)" /dev/mtd0ro
v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
v2.10.0 (release):OpenWrt v2024.01.17~bacca82a-3 (mt7622-snand-ubi-1ddr)
root@Belkin3:/tmp# hexdump -C /dev/mtd0 | grep -C 3 Built
00012590 65 6e 57 72 74 20 76 32 30 32 34 2e 30 31 2e 31 |enWrt v2024.01.1|
000125a0 37 7e 62 61 63 63 61 38 32 61 2d 33 20 28 6d 74 |7~bacca82a-3 (mt|
000125b0 37 36 32 32 2d 73 6e 61 6e 64 2d 75 62 69 2d 31 |7622-snand-ubi-1|
000125c0 64 64 72 29 00 42 75 69 6c 74 20 3a 20 32 32 3a |ddr).Built : 22:|
000125d0 34 38 3a 30 36 2c 20 41 75 67 20 31 32 20 32 30 |48:06, Aug 12 20|
000125e0 32 34 00 0a 42 4c 32 3a 20 46 61 69 6c 75 72 65 |24..BL2: Failure|
000125f0 20 69 6e 20 70 72 65 20 69 6d 61 67 65 20 6c 6f | in pre image lo|
--
00032590 65 6e 57 72 74 20 76 32 30 32 34 2e 30 31 2e 31 |enWrt v2024.01.1|
000325a0 37 7e 62 61 63 63 61 38 32 61 2d 33 20 28 6d 74 |7~bacca82a-3 (mt|
000325b0 37 36 32 32 2d 73 6e 61 6e 64 2d 75 62 69 2d 31 |7622-snand-ubi-1|
000325c0 64 64 72 29 00 42 75 69 6c 74 20 3a 20 32 32 3a |ddr).Built : 22:|
000325d0 34 38 3a 30 36 2c 20 41 75 67 20 31 32 20 32 30 |48:06, Aug 12 20|
000325e0 32 34 00 0a 42 4c 32 3a 20 46 61 69 6c 75 72 65 |24..BL2: Failure|
000325f0 20 69 6e 20 70 72 65 20 69 6d 61 67 65 20 6c 6f | in pre image lo|
--
00052590 65 6e 57 72 74 20 76 32 30 32 34 2e 30 31 2e 31 |enWrt v2024.01.1|
000525a0 37 7e 62 61 63 63 61 38 32 61 2d 33 20 28 6d 74 |7~bacca82a-3 (mt|
000525b0 37 36 32 32 2d 73 6e 61 6e 64 2d 75 62 69 2d 31 |7622-snand-ubi-1|
000525c0 64 64 72 29 00 42 75 69 6c 74 20 3a 20 32 32 3a |ddr).Built : 22:|
000525d0 34 38 3a 30 36 2c 20 41 75 67 20 31 32 20 32 30 |48:06, Aug 12 20|
000525e0 32 34 00 0a 42 4c 32 3a 20 46 61 69 6c 75 72 65 |24..BL2: Failure|
000525f0 20 69 6e 20 70 72 65 20 69 6d 61 67 65 20 6c 6f | in pre image lo|
--
00072590 65 6e 57 72 74 20 76 32 30 32 34 2e 30 31 2e 31 |enWrt v2024.01.1|
000725a0 37 7e 62 61 63 63 61 38 32 61 2d 33 20 28 6d 74 |7~bacca82a-3 (mt|
000725b0 37 36 32 32 2d 73 6e 61 6e 64 2d 75 62 69 2d 31 |7622-snand-ubi-1|
000725c0 64 64 72 29 00 42 75 69 6c 74 20 3a 20 32 32 3a |ddr).Built : 22:|
000725d0 34 38 3a 30 36 2c 20 41 75 67 20 31 32 20 32 30 |48:06, Aug 12 20|
000725e0 32 34 00 0a 42 4c 32 3a 20 46 61 69 6c 75 72 65 |24..BL2: Failure|
000725f0 20 69 6e 20 70 72 65 20 69 6d 61 67 65 20 6c 6f | in pre image lo|
4 Likes
I believe you might want to adjust the notes on this release to point users to v1.0.2 of the installer. It's currently suggesting users install v1.0.3.
https://github.com/dangowrt/owrt-ubi-installer/releases/tag/v1.1.1
2 Likes
It's a bit tricky as probably at this point you cannot easily install kmod-mtd-rw due to the snapshot being outdated and kernel dependencies hence no longer being met.
Which kernel version are you on right now?
it installs here with the snapshot provided by AUC
OpenWrt SNAPSHOT r27137-f51cb74473 / LuCI Master 24.212.79282~65b8002 6.6.45
1 Like
Worked exactly as described.
- First rewrite FIP in UBI via ubiupdatevol
- Did this and rebooted, no OKD
- Sysupgrade to new snapshot to get correct kmod-mtd-rw
- Built and upgraded to my own snapshot with kmod-mtd-rw built-in
- Use mtd to rewrite bl2 (preloader.bin)
- Went fine, confirmed update with grep and hexdump as previously described, rebooted fine
Thanks @hnyman for sharing how to get the fixes on the device.
Thanks all in this thread and especially @daniel for sticking with this device through 2024 to resolve OKD 
1 Like
If reading the entire FIP volume triggers scrubbing, it would be a lot safer to just read the entire FIP volume and dump it to /dev/null.
For those of us who have been on <=v1.0.2 the entire time, I assume we should probably be gentle on these devices until we move to snapshot/24.x? Reduce reboots, updates and anything that writes to NAND?
The low quality NAND combined with a lack of scrubbing and wear leveling in TF-A 2.4 piled on top of 2+ years of usage - aren’t we the most at risk now?
1 Like
I don't think it's a dire as you have stated. At least my 3 y/o E8450 is still churning along fine.
Note that bl2 is still located in MTD partition, so if that fails, router will be bricked as well.
I would say that the SoC's SNFI engine is doing it's job correcting bit flips. If anyone is concerned, I guess one can take the drastic step of re-writing the entire NAND flash once in a while to handle bit flips?
IMHO, I would think the NAND flash is operating as designed. The bug in TF-A 2.9 BL2 probably threw everyone off into chasing something that we may/should not be really concerned about.
9 Likes
I see that Installer v1.0.3 built files have been removed from https://github.com/dangowrt/owrt-ubi-installer/releases .
However the only change from v1.0.2 to v1.0.3 was a OpenWrt version bump from 22.03.x to 23.05.x series. There was no TF-A and U-Boot update based on a quick look.
Did something specific in OpenWrt 23.05.x onwards lead to more bit flips and trigger the OKD, or did 22.03.x users also report OKD?