Belkin RT3200/Linksys E8450 WiFi AX discussion

[Lynx]

What about:

@Lynx

... and especially the last paragraph.

So might it not help mitigate temperature-related read problems to alter the way in which the flash memory is read during boot, e.g. by avoiding reading from the same block too quickly?

Is there a way to modify the reads to avoid reading from the same block too quickly to potentially ameliorate read disturb?

[daniel]

That was also my first thought, and I implemented cache in the flash driver and to my surprise found out that there are never any cache hits -- because ubispl was modified by @hackpascal who had already taken care of that, so it already caches LEBs and thereby avoids repeatedly reading the same block.

[hnyman]

I continued my experiment after the OKD yesterday:

I compiled today the current main/master firmware and flashed it. Again the same condition: no LEDs and a dead router. A 40-second power off helped, and the router then booted ok settings intact.

LuCI stats shows that CPU(?) temp increased by some 3 degrees from 47 to 50 'C during the early sysupgrade process (checksum calculation etc. before the processes are killed and the actual flashing starts). The 40 second power-off helped to lower the temp to 45 'C, some 2 degrees below the previous steady running temp.

So, it was again the same: the actual flashing goes ok, but the flash read at the subsequent boot fails.

I seem to currently be in the lucky situation that the flash is written ok and no actual recovery actions are needed.

[jkdf2]

Quick question as someone who is having the same issue with semi-OKD: what setup are you using to preserve your stats between reboots?

[hnyman]

Just the normal (pretty recent) option in LuCI statistics standard RRD plugin config to enable backup
Available in main/master and 23.05.

Ps. the feature was introduced in Oct 2023, so pretty recent.

[BeardRobot]

I was wondering if anyone had created a 3D printed case for the RT3200/E8450 and came across this - https://www.thingiverse.com/thing:5864938
I might have to print me one!

[immi803]

Sir you can introduce new installer with fix for new users coming, thank you

[jkdf2]

Thanks, I have that option available on my install as well. So you try to avoid just flipping the switch or pulling the plug?

[quarky]

@ej_breaks_the_lan

I'd thought of that. But the dust blowing up into the case, the hassle of having to open it up periodically to clean out said dust, and vibration noise from the router sitting atop the spinning fan motivated me to try blowing air from the side.

Blowing at the case likely does nothing much to cool the internals of the router boards components as those are insulated by a blanket of air between it and the plastic case which you’re blowing air at.

Cool air gets sucked in from the bottom of the router and will move up thru heat sinks as air heated by the heat sinks and components rises thru the case’s top vents, so dust builds up regardless. I have one E8450 that lives permanently in a cabinet and I have a 120mm fan blowing up from its bottom. CPU temp hovers around 53C, so a 10C reduction without. The other 2 stays outside with adequate air flow, so no fans for those 2, heh heh. The idea is to move air over the router’s heat sinks and components.

If you’re worried about dust, get a simple dust filter and place it at the bottom in between the fan and router base air intake to minimize dust build up. But it will not completely stop it. Moving air create static electricity which will attract dust. So dust particles will build up regardless, unless your router is inside a clean room. Maybe get a slower blower to minimize the static effect.

I wouldn’t worry about it if I were you tho.

[quarky]

@hnyman

I compiled today the current main/master firmware and flashed it. Again the same condition: no LEDs and a dead router. A 40-second power off helped, and the router then booted ok settings intact.

LuCI stats shows that CPU(?) temp increased by some 3 degrees from 47 to 50 'C during the early sysupgrade process (checksum calculation etc. before the processes are killed and the actual flashing starts). The 40 second power-off helped to lower the temp to 45 'C, some 2 degrees below the previous steady running temp.

Somehow I’m extremely skeptical about temperature being a cause. My 3 E8450 would be seeing OKD all the time when any of them reboots. I’m not seeing that. All my E8450 are still running 5.15 master builds from end Jan 2024 master pulls with v1.0.2 or earlier installer.

I would suspect that maybe we’re seeing flash or memory caches not properly reset or something where after a power off, residual charges in the router board’s capacitors are still powering memory components. Waiting for the residual charges to dissipate likely resets all caches, and that also lowers the components temperature.

[Lynx]

So yet a further option then: not a temperature-related read error, but some kind of residual charge issue or so that causes a read error absent sufficient time or cooldown to reset.

That makes me think of this:

Buttonless Failsafe Mode

So, it turns out that this assumption -- that the only way to communicate information to the next boot without relying on flash is with the cooperation of the bootloader -- proves to be false. I grabbed a spare travel router of mine and hacked up a quick proof-of-concept to illustrate how: The idea is to reserve a single 4K page of RAM to stash some volatile information that can survive reboots. The page should be somewhere near the middle of the physical memory range, where the bootloader and…

where @CFSworks deftly showed how it is possible to store a value in RAM that survives warm reboots and can be checked for via checking a RAM-backed mtd device.

Could it be that there is some kind of reset line or function that needs to be called between reboots to clear any such residual charge issues?

@daniel

That was also my first thought, and I implemented cache in the flash driver and to my surprise found out that there are never any cache hits -- because ubispl was modified by @hackpascal who had already taken care of that, so it already caches LEBs and thereby avoids repeatedly reading the same block.

Ah - frustrating. Since this seemed to me promising in terms of a possible explanation for the issue given thef documents I identified about read disturb:

Belkin RT3200/Linksys E8450 WiFi AX discussion

That would be consistent with the freezer trick fixing the boot. And this did not happen before the new installer, right? So doesn't it mean there is something in the software or a hardware effect caused by the software, which suddenly amplifies the significance of the temperature of the router or flash chip specifically in respect of reads to the point that the router may not boot? Does UBI play a part here? I did a little digging and perhaps one or more of the following may help: https://…

And the first paper disclosed various techniques for mitigating like tweaking Vpass:

And calibration:

[rvs513]

Just for reference, I have been logging the router ‘temperature’ for several months.
I saw a ~10 degree drop when I switched to the on demand scheduler (the initial part of the graph). I also see a significant drop when WiFi is disabled. I use wifischedule to disable WiFi from 23h-06h. Here the average temperature drops from ~50 to ~40.
I have not had any boot issues so far.

[creo2005]

The 23.05.3 release was posted, but there is none for RT3200/E8450

[mdsharpe]

Check again now.
Disclaimer, many discourage downloading new releases before they are announced.

[dalutou]

Upgraded my E8450 to 23.05.3 just a moment ago, everything works fine.

I also upgrade bl2+uboot.fip to the new version through TFTP, because I need the patch mediatek: mt7622: linksys-e8450: set driving strength for SPI-NAND

fw_printenv ver
# old
ver=U-Boot 2023.07.02-OpenWrt-r23630-842932a63d (Nov 14 2023 - 13:38:11 +0000)
# new
ver=U-Boot 2023.07.02-OpenWrt-r23809-234f1a2efa (Mar 22 2024 - 22:09:42 +0000)

[el_charlie]

@dalutou

I also upgrade bl2+uboot.fip to the new version through TFTP, because I need the patch mediatek: mt7622: linksys-e8450: set driving strength for SPI-NAND

Can you post a guide how to do that?

I haven't updated the installer. I'm on stable 23.05.2 but used the old 1.0.1 installer for 22.03.x and then I sysupgraded to 23.05 when it came out.

[efahl]

@creo2005

The 23.05.3 release was posted

NO, it was not. A tag was created in the repo, in order for the devs to begin consolidating a release. There is no 23.05.3 release until an announcement is posted on the forums and announce list.

See A note about using new stable releases before they are announced

[dalutou]

Update the bootloader is a decision where risk outweighs reward.

I have bricked my router three times.

My suggestion is to wait for owrt v1.1.2 release.
Just reflash the installer and it will automatically upgrade the bootloader to the latest version.

If you really want to upgrade, it is recommended to use u-boot bootmenus.

Open your router, use the USB-Serial adapter to connect to the serial interfaces (RX, TX, GND). When the router power on, quickly press the down arrow and you will see bootmenus.

You can select the corresponding menu and proceed with the operation.

• Load BL31+U-Boot FIP via TFTP then write to flash
• Load BL2 preloader via TFTP then write to flash

Don't try these unless you know what you're doing, or you may brick your router.

[el_charlie]

Oh, I don't have an USB to serial adapter. Although I do have a TIAO USB Multiprotocol Adapter. Does that work?

Anyway. I prefer to run the updated installer when OpenWRT v24 arrives. Mine is working just fine with 600MHz min freq and I don't face reboot problems.

Cheers!