Troubleshooting Serial Connection Belkin RT3200

[sidus]

Hi there, I installed OpenWrt on my Belkin RT3200 (aka Linksys E8450) back in November 2023. Everything worked fine until a week ago. We lost power, so the router turned off and it did not boot back up like it usually would. The power LED does not go on and there is no ethernet connection. I suspected the OKD, as indicated in the wiki.

On to trouble shooting. I purchased a USB-to-Serial adapter, specifically this adapter, and connected the cables. I downloaded the mtk_uartboot, the bl2.bin, and the .fip files, then tried connecting. Here is the output:

> sudo ./mtk_uartboot -a -p bl2-for-mt7622_uartboot.bin -f *uboot.fip && screen /dev/ttyUSB0
mtk_uartboot - 0.1.1
Using serial port: /dev/ttyS15
thread 'main' panicked at src/main.rs:143:17:
Failed to open port: Error { kind: Unknown, description: "I/O error" }
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Aborted

It looks like the script is correctly identifying my serial adapter, but it isn't able to open a serial connection to my router.

Any ideas on what the issue is here? Do I have the wrong kind of serial adapter? Is my router toast? Is it some third thing I haven't thought of yet? Any help would be appreciated.

[grauerfuchs]

The error you're getting and the line number it provided suggests that something may have already taken control of the serial port that the cable uses. Check to make sure you don't have any terminal software already using the port. You may also need to disable the ModemManager service or its equivalent depending on your distribution, since it is known for hogging serial ports on Linux. You may need to unplug the adapter and plug it back in again as well.

[sidus]

Thanks for your recommendation. That wasn't exactly the issue, but it led me in the right direction. It seems the mtk_uartboot was auto-selecting the incorrect serial port. It defaulted to ttyS15, but I needed to direct it to ttyUSB0 instead by adding -s /dev/ttyUSB0 to the first command. After that, the serial interface completed the handshake, uploaded the payload, then the router booted and the power light went on.

I did run into a new issue though. When moving to boot, the serial interface displayed only garbage, as if it could not understand the character encoding. It was also unresponsive. I could not interrupt the boot to access the U-Boot menu or enter any commands. Once the device is booted, I can ssh in over ethernet though, so that is very promising.

[grauerfuchs]

Now that you mentioned having forgotten the -s /dev/ttyUSB0, that directed me to look at the rest of the command in detail. The reason you're only getting garbage after moving to boot is because you need to specify the speed when using screen. screen /dev/ttyUSB0 115200 will correct that.

[sidus]

Thanks for all your help! I got my router back up and running. It reboots and everything seems to function fine. Below are some of the problems I encountered and their fixes. For reference, my router was running 23.0.5.0 previously. If you were running a snapshot version, some of the steps below may be significantly different.

Problem: Belkin RT3200 (Linksys E8450) doesn't boot, no power or network connection lights, but hardware seems fine. Wiki indicates this is the OKD (OpenWRT kiss of death). From what I read, this seems to be because a bug was introduced in the boot process of v23.0.5.x causing previously correctable errors to become fatal.
Fix: Follow the steps listed in the wiki and discussed in threads on the forums. Specifically, connect to the serial console, access U-Boot recovery mode, and flash a boot preloader that isn't affected by the bug.

Problem: I used a USB-to-UART (TTL Serial) adapter, but the serial pins aren't seating properly.
Fix: My adapter had 2.54mm pitch female connectors (same size as FPIO and GPIO in most of my other devices), but the serial port on the RT3200 seems to use 2.0mm pitch male pins. I used test hook clips/leads to establish a secure connection to the existing pins. It's probably possible to find a correctly sized 6-pin female adapter as well. From what I read, it should be a 6-pin JST-PH connection, but I can't confirm this for certain as I used the test hook clips instead.

Problem: Running mtk_uartboot throws an IO error.
Fix: Make sure the program is using the correct serial port. In my case, the program was auto-selecting /dev/ttyS15, but I should have specified /dev/ttyUSB0. I had tricked myself, as I had initially specified that serial port for my screen command, but not for the mtk_uartboot command.

Problem: Running mt_uartbook establishes a connection and uploads the payload, but does not present the U-Boot Recovery Console. Instead, I see gibberish, then the router completes booting and enters a functional state, but still cannot boot up on its own in the future.
Fix: My screen tty was using the incorrect baud rate, and so the character encoding was all messed up. I had to manually specify the correct baud rate of 115200. Here is the full command I ended up running:

sudo ./mtk_uartboot -s /dev/ttyUSB0 -a -p ./mt7622-ram-1ddr-bl2.bin -f ./*uboot.fip && sudo screen /dev/ttyUSB0 115200

Problem: Next, instructions indicate I should Flash over TFTP a boot preloader that isn't bugged, but it isn't entirely clear what file I should use for this.
Fix: Discussion seems to indicate that reverting to a v22.0.3.x preloader is best. For now anyway. This will likely change once a newer stable version is released. I used the preloader from this page, as the v22.0.3.7 boot process isn't bugged.

Problem: Flash over TFTP doesn't seem to establish a connection.
Fix: In my case, I assumed my Lan 1 port would be used, but I needed to use the Wan port instead.

Problem: Flash over TFTP does not accept the preloader file, even though I am hosting it in the correct location and have established a network connection.
Fix: Make sure to rename the preloader file to openwrt-mediatek-mt7622-linksys_e8450-ubi-preloader.bin as indicated on the wiki. Flash over TFTP looks for that exact file name.

After all that, the correct preloader was flashed to my device and it could reboot without manual interference. Note, I am still running v23.0.5.0 of the OpenWrt firmware (yes, I plan to update it, that's next on my todo list). I only had to revert the boot preloader, not the entire installation.

[grauerfuchs]

No doubt some of the summary will be very helpful for other people that run into similar issues. Excellent job!

OKD was caused by a simple logic error in the v. 2.9 update for external TF-A package. TF-A provides the BL2 and BL3.1 components needed for this router to boot. In short, when the new SNFI driver was ported over to TF-A, the numeric output of the driver was not sanitized. The driver outputs a value which indicates success, uncorrectable error, or the count of corrected errors. TF-A expected a result that was exclusively Success (0) or Failure (non-0).

Yes, the serial header on the e8450/RT3200 is indeed a JST-PH (2,0mm pitch). Most USB UART adapters come with the 2,54mm pitch, since it is more common on devices that expect to use one.

The failure to establish a connection on LAN1 was probably a transient issue, potentially caused by lack of expected response or temporary lockout caused by a bad response. Every time I've run TFTP and connected to any e8450/RT3200, I had been using LAN1.