Asus RT-AC88U (HW: A6) broken in 22.03.3?

Installed 22.03.3 and router wouldn't boot. Had the power light on, after ~30s it would reboot, flash a few lights and then sit with the power light only on.

Was able to boot to recovery, reset NVRAM (no change), reflashed 22.03.3 (no change), flashed 22.03.2 and booted fine. Logged into Luci, flashed 22.03.3, same boot loop. Booted back to recovery, flashed 22.03.2 and restored backup - all is working well.

Notice this includes new wifi firmware, wondered if this could be the issue and if anyone else is having this issue - I'm running hardare A6.

Wonder, does the new wifi firmware need to be installed before upgrading? OpenWrt Support for Asus RT-AC88U - #283 by andyboeh

No, the firmware image is only required to get WiFi working. It is not flashed to the device but downloaded to the WiFi hardware whenever it is initialized.

I have the same problem, A6 hardware revision 22.03.3 fresh install does not boot, 22.03.2 works.

Hello !
I have the exact same problem as you do.

I just bought a 2nd hand rt-ac88u and have the same problem: not seeming to boot.
Flashed openwrt-22.03.3-bcm53xx-generic-asus_rt-ac88u-squashfs using the CFE miniWeb Server.

Version openwrt-22.03.2-bcm53xx-generic-asus_rt-ac88u-squashfs works fine (it boots).

I attached a serial console and made a log: https://pastebin.com/bMfNcq4t

Maybe someone understands what is going on. The last part:

iproc_nand 18028000.nand-controller: intfc status f0000000
iproc_nand 18028000.nand-controller: timeout waiting for command 0x1

seems to endlessly repeat.

I'm in the same boat, RT-AC88U A6 not working after attempt to upgrade to 22.03.3.

@arinc9 would you have a look at @wdehoog's log? Maybe you can figure out what went wrong? Did the kernel get to big? Finding the root failed?

On line 237 of @wdehoog log:

[    3.097181] UBI: auto-attach mtd4
[    3.100521] ubi0: attaching mtd4
[    3.120527] ubi0 error: 0xc04f0b3c: PEB 0 contains corrupted VID header, and the data does not contain all 0xFF
[    3.130664] ubi0 error: 0xc04f0b4c: this may be a non-UBI PEB or a severe VID header corruption which requires manual inspection

I've been aware of the latest 22.03 and master snapshots not working for a few months now. Something broke around this time where both me and the user on this post experienced it.

I thought it'd be fixed before a stable release. Thankfully @wdehoog bothered enough to rip out the monster plastic chasis so we can see the bootlog. I don't know much about the error but I can tell something is going wrong while trying to mount the filesystem from the flash.

Maybe @rmilecki can shed some light on this.

At least one can easily revert to 22.03.2 for the time being.

And someone please create an issue on GitHub with the bootlog.

There is already an issue on github: https://github.com/openwrt/openwrt/issues/11821.

I tried to bisect and it looks like one of the patches of https://github.com/openwrt/openwrt/commit/b66037f53bf3d943e23cad01986ba079cecb41cd is causing it.

No idea how I can proceed.

Patch 036-v5.19-0009 reverts changes from 036-v5.19-0002, but not from 036-v5.19-0003. This probably breaks nand-controller. Try to remove patch 036-v5.19-0003...and see if that fixes the problem.

I don't think that's related. Here's what @wdehoog should try:

Remove these two patches:
036-v5.19-0004-ARM-dts-BCM5301X-Add-rgmii-to-port-5-of-Broadcom-swi.patch
036-v5.19-0005-ARM-dts-BCM5301X-Retrieve-gmac1-MAC-address-from-NVR.patch

Recompile, see if it works.

At any rate, since you figured out this commit which includes these patches is the culprit, you can remove all the patches, confirm the device boots, then add a few patches back each time and see when it breaks.

@rincewind @arinc9

It looks like 036-v5.19-0005-ARM-dts-BCM5301X-Retrieve-gmac1-MAC-address-from-NVR.patch causes the problem.

But with only patches 1..4 even though the router boots the web interface only shows an empty directory list. The log: https://pastebin.com/tSd8NEj0

If you build from master branch, it doesn't have web interface enabled. You need to run "opkg update; opkg install luci" to get it.

I just tried this on an RT-AC88U, the changes on 036-v5.19-0005-ARM-dts-BCM5301X-Retrieve-gmac1-MAC-address-from-NVR.patch causes the issue. I'll investigate.

Although it may look like the change on 036-v5.19-0005-ARM-dts-BCM5301X-Retrieve-gmac1-MAC-address-from-NVR.patch breaks the system, the latest devicetree of this device works fine on older versions. The last version I tried is 22.03.0-rc1. I'm currently investigating what change actually causes this problem.

Update: It broke at 22.03.1. Something between 22.03.1 and 22.03.0 breaks it. I see a lot of mtd and nvmem commits from @rmilecki.

I just read a post in DD-Wrt forum regarding reset nvram during firmware upgrade may affect the stability of some Asus RT-AC routers (AC88U, AC3100, AC5300 were mentioned) because of AsusWRT changes. Would the same thing have implication to OpenWrt firmware? I was almost set to get a RT-AC88U to replace my EA7500v1 and now I think I should take a pause and see how it plays out.

The only driver I know on mainline Linux kernel that relies on NVRAM variables is the Broadcom wireless driver. It looks for how the wireless LEDs should function. Which I made OpenWrt set them at every boot. I'm aware DD-Wrt and AsusWrt relies heavily on NVRAM variables like switch configuration. This is not the case with OpenWrt since it sticks to mainline Linux with slight modifications. It's also worth mentioning that this is not related to the issue here at all.

Thanks for the insights on OpenWrt's nvram implementation strategy and good to know that there should be no impact to OpenWrt. I zoomed in on RT-AC88U mainly it the most 3rd-party firmware support and checked out DD-Wrt forum and FreshTomato's to get a feel how the different firmware fares before I commit to it. I am sure it should not take long for you fix the current problem on hands.

I've pinpointed the offending commit to:

@rmilecki looks like something you changed breaks nvmem. Or parsing nvmem-cells from dt binding weren't there before this commit, and now it's there but won't work properly.

Tested on the 22.03 branch, on the tree at this commit.

-Edir-
Oops. Sorry I did something wrong. It does not help.

It looks like this makes it work

nvram@1c080000 {
    compatible = "brcm,nvram", "nvmem-cells"

I realy have no idea why. This is all new to me but I saw other dts files have it and somewhere the mtd code checks for "nvmem-cells". I just tried it and it booted.