Edgerouter x bricked after sysupgrade applied

Hi, so here's my situation. I have an Edgerouter X lying around and wanted to check OpenWrt for security and performance bonuses over my TP-link on my home network. I had EdgeOS 2.0.9 hotfix2 running on ER-X, and I followed this guide to switch to OpenWrt. https://openwrt.org/toh/ubiquiti/edgerouter_x_er-x_ka

I flashed to OpenWrt (openwrt-ramips-mt7621-ubnt_edgerouter-x-initramfs-factory.tar) using the Web UI on the ER-X. After rebooting, I could SSH(via Putty) into OpenWrt on the router fine. I then used WinSCP to copy (openwrt-23.05.3-ramips-mt7621-ubnt_edgerouter-x-squashfs-sysupgrade.bin) to the /tmp/ folder on the router. I ran the sysupgrade command via SSH, but for some reason, the router will not boot properly now (no IP, etc.).

After some research, I pivoted to try the USB-TTL cable recovery option on the serial pins on the board to try to flash (Putty,tftp) a UI recovery image, but I didn't get anything in my Putty session. The screen stays blank, and I never get into a boot session at power on (interrupting by pressing the '1' key). If I can get that to connect, I hope to get a recovery image loaded back on the ER-X via tftp. I'm not sure where to turn to keep troubleshooting. If possible, I'd like to really take a deep dive into Openwrt to test some things.

Does anyone have extensive knowledge of serial TTL recovery on an ER-X? I want to get back to something usable. The joys of tinkering :slight_smile:

Try the EdgeRouter tftp recovery. Hopefully you have the bootloader upgraded to the latest.

Thanks for the post! Yep, got to that page through my initial research to get started and snag the recovery image. I followed a few folks videos to see if I had the serial pins correct. I'm pretty sure I've triple-checked my Putty serial settings, and when I power on the router when the Putty session is open, I get nothing. That's where I was hoping someone may have some insight or tips and tricks. If I can get that going, I'm pretty confident the tftp solution will push a recovery image. I'll keep tinkering and see if I can get the bootloader to start. Will update my post if I have success. Thanks again!

You don't need serial for the tftp recovery process (assuming you have an up-to-date bootloader).

As far as the UART connection, one thing to try is to short the tx and rx pins on your USB-ttl adapter... this creates a loopback connection and whatever you type should end up echo'd to the screen. If you don't see a working loopback, that means something isn't right with the adapter and/or your windows/putty configuration.

1 Like

That sounds good. I'll give this a whirl and see what I learn. I think the router is in some booting loop b/c the eth1 port lights up and flickers briefly, giving packet lights to the nic on the laptop, and then it goes dark again and starts all over. If I don't have a stable IP, I'm curious if/how tftp would be a solution for putting a recovery image on it? As for the bootloader status, that is anyone's guess. The ER-X has been in a drawer for over four years, so whatever was on it out of the box is its current state. I don't recall ever updating it. Thanks again for the tips.

So, the 'short' of Tx/Rx leads loopback works. I was able to see what I typed echo in the session window. The PL2303TA TTL cable appears to be working well, and I think my Putty settings are okay. I'll keep researching more in the coming days and see what I can learn. Thanks again.

If you were running 2.09 hf2, you very likely have the version 2 or 3 boot loader, which allows recovery without opening the case and connecting any pins. Don't connect pins - try easy first!

The Ubiquiti web-site has good directions. I won't repeat them here. You need to install and use a tftp server to recover though, not just putty. It's in the directions.

As for OpenWrt installation, this needs to be done with an initramfs image first, then you flash the permanent sysupgrade image. This post has a good summary for installation.

OpenWrt 23.05 runs rings around stock EdgeOS, which has essentially been abandoned by Ubiquiti anyway. I'd make the switch. Do install OpenWrt 23.05 though. 21.02 and 22.03 had some growing pains with the conversion to DSA.

EdgeOS has a version 3.0 on its way, but other than cosmetic changes and finally adding wireguard, it's same old same old.

1 Like

Well, lucky me, it worked like a charm on the first try. I'm back to a base image v2.0.6 on the ER-X. Why I went down the rabbit hole using UART and serial pins from a YouTube video, I'll never know. Anyway, I'm back in business and will explore your link to the post to get the Openwrt going. It doesn't look too crazy, so I'll report back with any comments/successes/failures. Thanks again for the kind reply!

1 Like

So, I'm sitting at Openwrt 22.03 SNAPSHOT and keep having issues when trying to push the squashfs-sysupgrade.bin file. I SCP the file to the tmp folder, SSH on OK, and when I run the sysuprade -n command to the /tmp/ folder of my .bin file, it errors and doesn't come back up after a reboot. So now I've flashed the ER-X recovery image a half-dozen times with tftp, upgraded to Openwrt, and got to the part to run sysupgrade with no dice. I'm sure there may be different sysupgrade files. I used the ones here https://github.com/stman/OpenWRT-19.07.2-factory-tar-file-for-Ubiquiti-EdgeRouter-x/tree/master/Version%2022.03. I'll put this to bed for a little while and see what I can learn after some more research.

I am having a really bad case of deja vu dman34usa! And I'm laughing with you, not at you. Really. Read the post I linked again for the OpenWrt flashing directions. That's me in there having this exact issue.

Next time do this:

  1. start after a clean reboot to 2.06 EdgeOS the recovery image
  2. After you load the initramfs OpenWRt image to memory and it warm boots into it:
    a. scp the permanent sysupgrade image to /tmp as you have before
    b. ssh into the ER-X and cd /tmp
    c. sysupgrade -F -n /tmp/[sysupgrade image]
    d. Go get a beer. Do some laundry. Heck, go for a hike. But whatever you do, don't touch anything for a long time!

Hopefully you'll come back to an ER-X with OpenWrt running on it. I'm not going to admit how many times I went through this process convinced the final flash didn't take, because it was taking so darn long, pulled the plug, and went through the Ubiquiti tftp recovery again. But it sounds like you're getting close :wink:

Good luck.

1 Like

After you load the 22.03 initramfs image, use this sysupgrade file:

You start with an intramfs image small enough to fit in and boot from volatile memory - that is why you need the special initramfs image, not the one in the OpenWrt official downloads - but that will be gone if you reboot your device. However, the final flash is permanent (into your flash chip), so do it with a regular official OpenWrt sysupgrade image.

Ok, that sounds good! I'll show some patience. So, I ssh'd into the Openwrt (snapshot) after copying via scp the upgrade image you recommended to the /tmp folder. I ran the sysupgrade -F -n command, which gives 'Commencing upgrade. Closing all shell sessions. Command failed: Connection failed.' So, I'm not sure if that's expected behavior, but I'll go take a nap for a while since I got off work early for the holiday and come back and check. Thanks again for the kind response.

Well, I'll be a son of a sea cook (mom's favorite phrase), so I'm in. Thanks for the simple offer of 'patience Padawan' :slight_smile: I'll have to give it a test drive here and then maybe throw it on my ER-4, which is the brain of my home network. Thanks again! Take care.

Glad to hear you got through it. Now you just need to install some packages to add whatever capabilities you are interested in. Here's a short list to get you thinking:

opkg update
opkg install luci-app-sqm luci-proto-wireguard curl ca-certificates luci-app-https-dns-proxy luci-app-statistics luci-app-adblock tcpdump-mini openvpn-mbedtls luci-app-openvpn openssh-sftp-server iperf3 irqbalance htop luci-app-pbr

Next time you sysupgrade to a new release, check out the firmware selector. There you can add more packages to the customize packages drop down menu so they will already be included after you flash the firmware.

1 Like