Reboot on stalled on WRT3200ACM?

Hi, I have installed OpenWRT on both partitions of a WRT3200ACM and it behaveced very unreliable, when I put it in my network:

  • wouldn't reboot
  • wouldn't remember the settings

So I opened the case, attached a serail connector to the uart output and booted.
That seemed fine, until i tried a reboot and now I am stuck at:

...
[  114.442819] br-lan: port 3(lan3) entered disabled state
[  114.579852] device lan4 left promiscuous mode
[  114.584281] br-lan: port 4(lan4) entered disabled state
[  114.829156] device eth0 left promiscuous mode
[  114.837532] mvneta f1070000.ethernet eth0: Link is Down
[  115.949601] UBIFS (ubi1:0): un-mount UBI device 1
[  115.954414] UBIFS (ubi1:0): background thread "ubifs_bgt1_0" stops
[  118.969118] reboot: Restarting system

Is it my serial adapter that keeps the access point from rebooting, or am I already looking at a possible reason for the unreliability of the AP?

Ok, it looks like it is stuck unless i remove the usb-to-ttl adapter, but once I remove it and plug it back in, then it moves forward.
But it seems to halt in U-Boot, and not load the Openwrt root partition. Any idea what could cause this?

Marvell>> version                                                                                                         
U-Boot 2013.01 (Sep 25 2017 - 11:42:52) Marvell version: 2015_T1.QA.0p16

Boot version : v1.0.0
arm-marvell-linux-gnueabi-gcc (Linaro GCC branch-4.6.4. Marvell GCC 201301-1645.aee66e26) 4.6.4 20120731 (prerelease)
GNU ld (Linaro GCC branch-4.6.4. Marvell GCC 201301-1645.aee66e26) 2.22.0.20120801
Marvell>>

Switch partitions?

Do you mean with the button? Or is there a command for that?

I found printenv so far, maybe this has some info?

Marvell>> printenv
CASset=max
MALLOC_len=5
MPmode=SMP
SMT-2D=NIFNI480048EX01
altFwSize=0x5000000
altKernAddr=0x5a00000
altKernSize=0x0600000
altnandboot=setenv bootargs console=ttyS0,115200 root=/dev/mtdblock8 ro rootdelay=1 rootfstype=jffs2 earlyprintk $mtdparts                                                                                                                            ;nand read $defaultLoadAddr $altKernAddr $altKernSize; bootm $defaultLoadAddr
auto_recovery=yes
autoload=no
baudrate=115200
boot_order=hd_scr usb_scr mmc_scr hd_img usb_img mmc_img pxe net_img net_scr
boot_part=1
boot_part_ready=3
bootargs_dflt=$console $nandEcc $mtdparts_lgcy $bootargs_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_en                                                                                                                            d $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_panel=$lcd_panel
bootargs_end=:10.4.50.254:255.255.255.0:Armada38x:eth0:none
bootargs_root=root=/dev/nfs rw
bootcmd=run nandboot
bootcmd_auto=stage_boot $boot_order
bootcmd_fdt=tftpboot 0x2000000 $image_name;tftpboot $fdtaddr $fdtfile;setenv bootargs $console $nandEcc $mtdparts $bootarg                                                                                                                            s_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_end $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0                                                                                                                            _enable=$lcd0_enable clcd.lcd_panel=$lcd_panel; bootz 0x2000000 - $fdtaddr;
bootcmd_fdt_boot=tftpboot 0x2000000 $image_name; setenv bootargs $console $nandEcc $mtdparts $bootargs_root nfsroot=$serve                                                                                                                            rip:$rootpath ip=$ipaddr:$serverip$bootargs_end $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable                                                                                                                             clcd.lcd_panel=$lcd_panel; bootz 0x2000000 - $fdtaddr;
bootcmd_fdt_edit=tftpboot $fdtaddr $fdtfile; fdt addr $fdtaddr; setenv bootcmd $bootcmd_fdt_boot
bootcmd_lgcy=tftpboot 0x2000000 $image_name;setenv bootargs $console $nandEcc $mtdparts_lgcy $bootargs_root nfsroot=$serve                                                                                                                            rip:$rootpath ip=$ipaddr:$serverip$bootargs_end  video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_pan                                                                                                                            el=$lcd_panel;  bootm $loadaddr;
bootdelay=3
cacheShare=no
console=console=ttyS0,115200
defaultLoadAddr=0x2000000
device_partition=0:1
disaMvPnp=no
eeeEnable=no
enaClockGating=no
enaCpuStream=no
enaFPU=yes
enaMonExt=no
enaWrAllo=no
eth1addr=24:F5:A2:C0:F4:88
eth1mtu=1500
eth2addr=24:F5:A2:C0:F4:88
eth2mtu=1500
eth3addr=24:F5:A2:C0:F4:88
eth3mtu=1500
ethact=egiga0
ethaddr=24:F5:A2:C0:F4:88
ethmtu=1500
ethprime=egiga0
fdt_addr=2040000
fdt_skip_update=no
fdtaddr=0x1000000
fdtfile=armada-38x-modular.dtb
firmwareName=rango.img
flash_alt_image=tftpboot $defaultLoadAddr $firmwareName && nand erase $altKernAddr $altFwSize && nand write $defaultLoadAd                                                                                                                            dr $altKernAddr $filesize
flash_pri_image=tftpboot $defaultLoadAddr $firmwareName && nand erase $priKernAddr $priFwSize && nand write $defaultLoadAd                                                                                                                            dr $priKernAddr $filesize
ide_path=/
image_name=uImage
initrd_name=uInitrd
ipaddr=192.168.1.1
kernel_addr_r=2080000
lcd0_enable=0
lcd0_params=640x480-16@60
lcd_panel=0
limit_dram_size=yes
loadaddr=0x02000000
loads_echo=0
mtddevname=uboot
mtddevnum=0
mtdids=nand0=armada-nand
mtdparts=mtdparts=armada-nand:2048K(uboot)ro,128K(u_env),256K(s_env),256K@8064K(devinfo),1920K@8320K(sysdiag),80m@10m(kern                                                                                                                            el),74m@16m(rootfs),80m@90m(alt_kernel),74m@96m(alt_rootfs),160m@10m(ubifs),-@170m(syscfg)
mvNetConfig=mv_net_config=4,(00:50:43:11:11:11,0:1:2:3),mtu=1500
mv_pon_addr=00:50:43:db:f5:00
nandEcc=nfcConfig=4bitecc
nandboot=setenv bootargs console=ttyS0,115200 root=/dev/mtdblock6 ro rootdelay=1 rootfstype=jffs2 earlyprintk $mtdparts;na                                                                                                                            nd read $defaultLoadAddr $priKernAddr $priKernSize; bootm $defaultLoadAddr
netbsd_en=no
netmask=255.255.255.0
netretry=no
partition=nand0,0
pcieTune=no
pexMode=RC
priFwSize=0x5000000
priKernAddr=0x0a00000
priKernSize=0x0600000
pxe_files_load=:default.arm-armadaxp-db:default.arm-armadaxp:default.arm
pxefile_addr_r=3100000
ramdisk_addr_r=2880000
rootpath=/srv/nfs/
run_script=no
sata_delay_reset=0
sata_dma_mode=yes
script_addr_r=3000000
script_name=boot.scr
sd_detection_dat3=no
serverip=192.168.1.254
standalone=fsload 0x2000000 $image_name;setenv bootargs $console $nandEcc $mtdparts_lgcy root=/dev/mtdblock0 rw ip=$ipaddr                                                                                                                            :$serverip$bootargs_end; bootm 0x2000000;
stderr=serial
stdin=serial
stdout=serial
update_both_images=tftpboot $defaultLoadAddr $firmwareName && nand erase $priKernAddr $priFwSize && nand erase $altKernAdd                                                                                                                            r $altFwSize && nand write $defaultLoadAddr $priKernAddr $filesize && nand write $defaultLoadAddr $altKernAddr $filesize
usb0Mode=host
usbActive=0
usbType=2
vxworks_en=no
yuk_ethaddr=00:00:00:EE:51:81

Environment size: 4571/131068 bytes
Marvell>>

power button to see if the other partition is viable, but since you have serial just flash a known good to you current partition.

out of interest, what version were you flashing when this broke?

@ghoffman
I flashed 23.05.5 to both partitions a few days ago and had 22.x.x on it for a few month about a year or two ago. I try to use it in a different project. It used to be a little bit flakey in the past, but generally worked. It was attached via powerlan so I thought that was the reason.

@anomeome ah I see, I figured I could use the serial method as well.
and both run nandboot and run altnandbootwill boot into OpenWRT, but I am not sure it did really load different images:

First try:

BusyBox v1.36.1 (2024-09-23 12:34:46 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.5M      4.5M         0 100% /rom
tmpfs                   248.0M     60.0K    248.0M   0% /tmp
/dev/ubi0_1              55.4M    212.0K     52.4M   0% /overlay
overlayfs:/overlay       55.4M    212.0K     52.4M   0% /
ubi1:syscfg              70.2M    488.0K     66.1M   1% /tmp/syscfg
tmpfs                   512.0K         0    512.0K   0% /dev

Second try:

BusyBox v1.36.1 (2024-09-23 12:34:46 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 4.5M      4.5M         0 100% /rom
tmpfs                   248.0M     60.0K    248.0M   0% /tmp
/dev/ubi0_1              55.4M    204.0K     52.4M   0% /overlay
overlayfs:/overlay       55.4M    204.0K     52.4M   0% /
ubi1:syscfg              70.2M    488.0K     66.1M   1% /tmp/syscfg
tmpfs                   512.0K         0    512.0K   0% /dev

Should the /overlay devices be the same? Probably not, right?

I always seem to end up in Partition 1.
Tried /usr/sbin/fw_setenv boot_part 2 && reboot as well, but it didn't help, so maybe U-Boot and the two partitions are broken?

What would be the best way to reset U-Boot as well?

Seems OK

root@bsaedgy:~# cat /proc/mtd 
dev:    size   erasesize  name
mtd0: 00200000 00020000 "u-boot"
mtd1: 00020000 00020000 "u_env"
mtd2: 00040000 00020000 "s_env"
mtd3: 00040000 00020000 "devinfo"
mtd4: 001e0000 00020000 "sysdiag"
mtd5: 05000000 00020000 "kernel1"
mtd6: 04a00000 00020000 "rootfs1"
mtd7: 05000000 00020000 "kernel2"
mtd8: 04a00000 00020000 "ubi"
mtd9: 05600000 00020000 "syscfg"
mtd10: 005c0000 00020000 "unused_area"

script

bootTaOther
#!/bin/sh
#hacked from /lib/upgrade/linksys.sh

cur_boot_part=`/usr/sbin/fw_printenv -n boot_part`
target_firmware=""
if [ "$cur_boot_part" = "1" ]
then
    target_firmware="kernel2"
    fw_setenv boot_part 2
    fw_setenv bootcmd "run altnandboot"
elif [ "$cur_boot_part" = "2" ]
then
    target_firmware="kernel1"
    fw_setenv boot_part 1
    fw_setenv bootcmd "run nandboot"
fi
# re-enable recovery so we get back if the new firmware is broken
fw_setenv auto_recovery yes
echo "$target_firmware"
reboot

I have been running run nandboot and run altnandboot fro U-Boot by hand, I think wrapping a script around it probably wont make a difference.

Marvell>> run altnandboot

NAND read: device 0 offset 0x5a00000, size 0x600000
 6291456 bytes read: OK
## Booting kernel from Legacy Image at 02000000 ...
   Image Name:   ARM OpenWrt Linux-5.15.167
   Created:      2024-09-23  12:34:46 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3456053 Bytes = 3.3 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

Starting kernel ...

and

Marvell>> run nandboot

NAND read: device 0 offset 0xa00000, size 0x600000
 6291456 bytes read: OK
## Booting kernel from Legacy Image at 02000000 ...
   Image Name:   ARM OpenWrt Linux-5.15.167
   Created:      2024-09-23  12:34:46 UTC
   Image Type:   ARM Linux Kernel Image (uncompressed)
   Data Size:    3456053 Bytes = 3.3 MiB
   Load Address: 00008000
   Entry Point:  00008000
   Verifying Checksum ... OK
   Loading Kernel Image ... OK
OK

What I am seeing is that they read from different offsets ( 0x5a00000 vs 0xa00000 ) and after another try I seem to get into the second partition:

BusyBox v1.36.1 (2024-09-23 12:34:46 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:/# /usr/sbin/fw_printenv -n boot_part
2

I assume this means that this is a U-Boot issue and not an image issue, because unless I do run nandboot nothing happens, right?

As far as I see you are successfully booting each partition. Can you not save changes to one / both, one / both do not reboot on their own. Any error in bootlog on initial boot:
dmesg | grep -i err

I think I don't fully understand, what you mean.
My problem is: no partition at all boots on its own. U-Boot seems to stall. I think dmesg would show only messages after the kernel was selected and already started, right?

If you think uboot env is borked try reset and save

From your OP you said cannot save, will not reboot. I see kernel2 boot in the above post. If just partition 1 is borked, maybe try flashing an image, whether by tftp with uboot run flash_pri_image, or from partition 2 image.

I tried a reset of the environment variables of U-Boot, but this didn't work either. It looks like my WRT3200ACM is stuck in U-boot without the boot command being run.

I would provide a dump of my rango uboot env to compare but do not have a serial connected to that device; maybe @ghoffman if currently wired.

i think these are the symptoms that I have described for borked sysupgrade on venom.
i used the 'workaround' of force-flashing the 'factory' openwrt image for 23.05.5. to both partitions using the following commands (links changed for wrrt3200)
`
cd /tmp
wget https://downloads.openwrt.org/releases/23.05.5/targets/mvebu/cortexa9/openwrt-23.05.5-mvebu-cortexa9-linksys_wrt3200acm-squashfs-factory.img

mtd erase kernel1
mtd write openwrt...factory.img kernel1
mtd erase kernel2
mtd write openwrt...factory.img kernel2
reboot
`
this gets both partitions initially flashed.
i still generally have to sysupgrade twice, which efectively gets me back to the original partition. any changes to the defaults arent saved with the first sysupgrade, so you end up on the 2nd partition with default values. sysupgrading again gets yo back to partition 1, where your changed variables are still rpesent.

as I and others have documented, this has been a problem for the mvebu wrt sereis for about 4 months now.
the issue is documented throughout the forum and here:

but it is still not being picked up for investigation

2 Likes

@ghoffman , @anomeome .

thanks for your help, I erased the partitions and set them up again.
At the moment it works again.
My takeaways:

  • Debugging the boot process via serial connection does not really work, as it keeps the WRT3200ACM from booting
  • Whatever happened the last time I tried may have just been misconfiguration of one of the partitions
  • The configuration is not shared between partitions but must be set up separately for each

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.