How to recover rootfs on U-Boot based appliance

Hello everyone,

I am using QNAP QSW-M1208-8C 10GbE managed switch.
This product is based on U-Boot and OpenWrt (but source code may be unavailable at this time).

Several days ago, I updated firmware of QSW-M1208-8C from v1.0.0 to v1.2.0.
Then, QSW-M1208-8C has gone silent.
Power LED light up, but LAN LED does not light up and LAN port does not link up.
Only maintenance port can link up and light up.
After that, I found the following in release note.

Make sure you update the firmware incrementally (e.g., from 1.0 to 1.1.0) rather than updating directly to the latest version (e.g., from 1.0 to 1.2.0).

Now I am trying to fix this problem via serial console.
My question is "How can I fix this problem?".
The following is my trial records.

First, I connected my PC to QSW-M1208-8C using USB-serial converter and tried to access to the serial console based on the following setting.

  • Baud rate: 115200
  • Data bit: 8
  • Parity: none
  • Stop bit: 1
  • Flow control: none

My PC has been succeccfully connected to the serial console and found that the following message.

Wrong Image Format for bootm command
ERROR: can't get kernel image!

The boot messages of Marvell U-Boot is the following.

BootROM - 1.73
Booting from NAND flash

General initialization - Version: 1.0.0
AVS selection from EFUSE disabled (Skip reading EFUSE values)
Overriding default AVS value to: 0x23
Detected Device ID 6811
High speed PHY - Version: 2.0

Init Customer board board SerDes lanes topology details:
 | Lane # | Speed|    Type     |
 ------------------------------|
 |   1    |  5   |  PCIe0      |
 |   2    |  0   |  SGMII1     |
 -------------------------------
:** Link is Gen1, check the EP capability 
PCIe, Idx 0: Link upgraded to Gen2 based on client cpabilities 
High speed PHY - Ended Successfully
DDR3 Training Sequence - Ver TIP-1.55.0
DDR3 Training Sequence - Switching XBAR Window to FastPath Window 
DDR3 Training Sequence - Ended Successfully
BootROM: Image checksum verification PASSED

 __   __                      _ _
|  \/  | __ _ _ ____   _____| | |
| |\/| |/ _` | '__\ \ / / _ \ | |
| |  | | (_| | |   \ V /  __/ | |
|_|  |_|\__,_|_|    \_/ \___|_|_|
         _   _     ____              _
        | | | |   | __ )  ___   ___ | |_ 
        | | | |___|  _ \ / _ \ / _ \| __| 
        | |_| |___| |_) | (_) | (_) | |_ 
         \___/    |____/ \___/ \___/ \__| 
 ** LOADER **


U-Boot 2013.01 (Dec 04 2019 - 06:20:33) Marvell version: 2016_T1.0.eng_drop_v10

Board: A38x-QNAP-Board_Joe_V1
SoC:   MV88F6811 Rev A0
       running 2 CPUs
CPU:   ARM Cortex A9 MPCore (Rev 1) LE
       CPU 0
       CPU    @ 1332 [MHz]
       L2     @ 666 [MHz]
       TClock @ 200 [MHz]
       DDR3    @ 666 [MHz]
       DDR3 16 Bit Width,FastPath Memory Access, DLB Enabled, ECC Disabled
DRAM:  512 MiB
NAND:  nand deviceId = 0xdc2c  
512 MiB
MMC:   mv_sdh: 0
PCI-e 0 (IF 0 - bus 1) Root Complex Interface, Detected Link X1, GEN 2.0
USB3.0 0: Host Mode
USB3.0 1: Host Mode

Map:   Code:                    0x1fece000:0x1ff953e0
       BSS:                     0x1ffef6ec
       Stack:                   0x1f9cdf20
       Heap:                    0x1f9ce000:0x1fece000
       U-Boot Environment:      0x00500000:0x00580000 (NAND)

Board configuration detected:
Error: PHY Address of Port 0 (RGMII) is invalid (-1).
Net:   Error: PHY Address of Port 0 (RGMII) is invalid (-1).

|  port  | Interface | PHY address  |
|--------|-----------|--------------|
| egiga1 |   SGMII   |     0x00     |
Error: PHY Address of Port 0 (RGMII) is invalid (-1).
, egiga1
Hit any key to stop autoboot:

The following is output of "mtdparts" in U-Boot.

device nand0 <pxa3xx_nand-0>, # parts = 8
 #: name                size            offset          mask_flags
 0: u-boot              0x000000500000          0x000000000000          1
 1: uboot_env           0x000000080000          0x000000500000          0
 2: padding             0x000000500000          0x000000580000          0
 3: kernel1             0x00000de00000          0x000000a80000          0
 4: rootfs1             0x00000d800000          0x000001080000          0
 5: kernel2             0x00000de00000          0x00000e880000          0
 6: rootfs2             0x00000d800000          0x00000ee80000          0
 7: syscfg              0x000003700000          0x00001c680000          0

active partition: nand0,0 - (u-boot) 0x000000500000 @ 0x000000000000

defaults:
mtdids  : none
mtdparts: none

The following is output of "printenv" in U-Boot.

CASset=max
MALLOC_len=5
MPmode=SMP
auto_recovery=yes
autoload=no
altnandboot=run blink_status_led;setenv bootargs console=ttyS0,115200 ubi.mtd=6 root=/dev/ubiblock0_0 rootfstype=squashfs;nand read 0x2000000 0xe880000 0x33C0000;bootm
baudrate=115200
blink_status_led=mw.b $loadaddr 1;i2c write $loadaddr 26 8 1;
boot_order=hd_scr usb_scr mmc_scr hd_img usb_img mmc_img pxe net_img net_scr
boot_part=2
bootargs=console=ttyS0,115200 ubi.mtd=6 root=/dev/ubiblock0_0 rootfstype=squashfs
bootargs_dflt=$console $nandEcc $mtdparts_lgcy $bootargs_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_end $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_panel=$lcd_panel
bootargs_end=:10.4.50.254:255.255.255.0:Armada38x:eth0:none
bootargs_root=root=/dev/nfs rw
bootcmd=run altnandboot
bootcmd_auto=stage_boot $boot_order
bootcmd_fdt=tftpboot 0x2000000 $image_name;tftpboot $fdtaddr $fdtfile;setenv bootargs $console $nandEcc $mtdparts $bootargs_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_end $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_panel=$lcd_panel; bootz 0x2000000 - $fdtaddr;
bootcmd_fdt_boot=tftpboot 0x2000000 $image_name; setenv bootargs $console $nandEcc $mtdparts $bootargs_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_end $mvNetConfig video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_panel=$lcd_panel; bootz 0x2000000 - $fdtaddr;
bootcmd_fdt_edit=tftpboot $fdtaddr $fdtfile; fdt addr $fdtaddr; setenv bootcmd $bootcmd_fdt_boot
bootcmd_lgcy=tftpboot 0x2000000 $image_name;setenv bootargs $console $nandEcc $mtdparts_lgcy $bootargs_root nfsroot=$serverip:$rootpath ip=$ipaddr:$serverip$bootargs_end  video=dovefb:lcd0:$lcd0_params clcd.lcd0_enable=$lcd0_enable clcd.lcd_panel=$lcd_panel;  bootm $loadaddr; 
bootdelay=3
cacheShare=no
console=console=ttyS0,115200
device_partition=0:1
disaMvPnp=no
eeeEnable=no
enaClockGating=no
enaCpuStream=no
enaFPU=yes
enaMonExt=no
enaWrAllo=no
eth1addr=snip
eth1mtu=1500
eth2addr=snip
eth2mtu=1500
eth3addr=snip
eth3mtu=1500
ethact=egiga1
ethaddr=snip
ethmtu=1500
ethprime=egiga0
fdt_addr=2040000
fdt_skip_update=yes
fdtaddr=0x1000000
fdtfile=armada-38x.dtb
fileaddr=2000000
filesize=33C0000
ide_path=/
ilen=33C0000
image_name=zImage
imgname=QSW-ALD1200-FW.v1.0.0_S20200910_16817.bin
initrd_name=uInitrd
ioffset=0xa80000
ipaddr=169.254.100.1
kernel_addr_r=2080000
lcd0_enable=0
lcd0_params=640x480-16@60
lcd_panel=0
limit_dram_size=yes
loadaddr=0x02000000
loads_echo=0
mtddevname=u-boot
mtddevnum=0
mtdids=nand0=pxa3xx_nand-0
mtdparts=mtdparts=pxa3xx_nand-0:5M(u-boot)ro,512k(uboot_env),5M(padding),222M(kernel1),216M@0x1080000(rootfs1),222M(kernel2),216M@0xEE80000(rootfs2),55M(syscfg)
mtdparts_lgcy=mtdparts=armada-nand:8m(boot)ro,8m@8m(kernel),-(rootfs);pxa3xx_nand-0:8m(boot)ro,8m@8m(kernel),-(rootfs)
mvNetConfig=mv_net_config=4,(snip),mtu=1500
mv_pon_addr=snip
nandEcc=nfcConfig=8bitecc
nandboot=run blink_status_led;setenv bootargs console=ttyS0,115200 ubi.mtd=4 root=/dev/ubiblock0_0 rootfstype=squashfs;nand read 0x02000000 0xa80000 0x33C0000;bootm
net2nandp1=set ioffset 0xa80000;tftpboot $imgname;set ilen $filesize;nand erase.part kernel1;nand erase.part rootfs1;nand write $loadaddr $ioffset $ilen
net2nandp2=set ioffset 0xe880000;tftpboot $imgname;set ilen $filesize;nand erase.part kernel2;nand erase.part rootfs2;nand write $loadaddr $ioffset $ilen
netbsd_en=no
netmask=255.255.255.0
netretry=no
partition=nand0,0
pcieTune=no
pexMode=RC
pxe_files_load=:default.arm-armadaxp-db:default.arm-armadaxp:default.arm
pxefile_addr_r=3100000
qbootmode=normal
ramdisk_addr_r=2880000
rootpath=/srv/nfs/
run_script=no
sata_delay_reset=0
sata_dma_mode=yes
script_addr_r=3000000
script_name=boot.scr
sd_detection_dat3=no
serverip=169.254.100.177
skip_switch_pci_scan=no
stderr=serial
stdin=serial
stdout=serial
usb0Mode=host
usbActive=0
usbType=2
vxworks_en=no
yuk_ethaddr=snip

QSW-M1208-8C has "maintenance port" (RJ-45) and this has been still alive.
I connected maintenance port to intranet router using CAT6 cable, and executed "dhcp".
QSW-M1208-8C successfully got IP from router.
I also prepared TFTP server and placed "kernel" (kernel image) and "root" (squashfs rootfs image) extracted from firmware upgrade image provided at the following URL by QNAP in TFTP server.

Note that firmware upgrade image provided by QNAP is "sysupgrade" image and can extract by tar.
Then, I executed the following step in U-Boot console.

# set TFTP server IP
setenv serverip IP_OF_TFTPSERVER
# get kernel image from TFTP server
tftp kernel
# erase kernel1 partition
nand erase.part kernel1
# write kernel image to kernel1 partition
nand write 0x2000000 0xa80000 0x2390ac
# erase kernel2 partition
nand erase.part kernel2
# write kernel image to kernel2 partition
nand write 0x2000000 0xe880000 0x2390ac
# get rootfs image from TFTP server
tftp root
# erase rootfs1 partition
nand erase.part rootfs1
# make UBI partition in rootfs1
ubi part nand0,4
ubi create rootfs1
# write rootfs image to UBI partition
ubi write 0x2000000 rootfs1 0x2be191c
# erase rootfs2 partition
nand erase.part rootfs2
# make UBI partition in rootfs2
ubi part nand0,6
ubi create rootfs2
# write rootfs image to UBI partition
ubi write 0x2000000 rootfs2 0x2be191c

(In this step, I destroyed backup partition by mistake)
(Write size must be adjusted to the image size)

Finally, I executed "boot" and got the following error.

[    1.179450] ubi0: attaching mtd6
[    1.186120] random: fast init done
[    1.274139] ata2: SATA link down (SStatus 0 SControl 300)
[    1.279602] ata1: SATA link down (SStatus 0 SControl 300)
[    1.442610] random: crng init done
[    1.725025] ubi0: scanning is finished
[    1.733246] ubi0 warning: ubi_eba_init: cannot reserve enough PEBs for bad PEB handling, reserved 8, need 40
[    1.743453] ubi0: attached mtd6 (name "rootfs2", size 216 MiB)
[    1.749319] ubi0: PEB size: 262144 bytes (256 KiB), LEB size: 253952 bytes
[    1.756239] ubi0: min./max. I/O unit sizes: 4096/4096, sub-page size 4096
[    1.763067] ubi0: VID header offset: 4096 (aligned 4096), data offset: 8192
[    1.770064] ubi0: good PEBs: 864, bad PEBs: 0, corrupted PEBs: 0
[    1.776105] ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128
[    1.783368] ubi0: max/mean erase counter: 4/1, WL threshold: 4096, image sequence number: 0
[    1.791762] ubi0: available PEBs: 0, total reserved PEBs: 864, PEBs reserved for bad PEB handling: 8
[    1.800951] ubi0: background thread "ubi_bgt0d" started, PID 592
[    1.801174] mvsw61xx 10.mvsw61xx: No compatible switch found at 0x10
[    1.813508] armada38x-rtc f10a3800.rtc: setting system clock to 2022-02-11 12:40:07 UTC (1644583207)
[    1.823091] VFS: Cannot open root device "ubiblock0_0" or unknown-block(0,0): error -6
[    1.831049] Please append a correct "root=" boot option; here are the available partitions:
[    1.839460] 1f00            5120 mtdblock0 
[    1.839463]  (driver?)
[    1.846042] 1f01             512 mtdblock1 
[    1.846044]  (driver?)
[    1.852616] 1f02            5120 mtdblock2 
[    1.852617]  (driver?)
[    1.859198] 1f03          227328 mtdblock3 
[    1.859200]  (driver?)
[    1.865776] 1f04          221184 mtdblock4 
[    1.865777]  (driver?)
[    1.872349] 1f05          227328 mtdblock5 
[    1.872350]  (driver?)
[    1.878928] 1f06          221184 mtdblock6 
[    1.878929]  (driver?)
[    1.885505] 1f07           56320 mtdblock7 
[    1.885506]  (driver?)
[    1.892078] Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
[    1.900386] CPU1: stopping
[    1.903109] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 4.14.158 #0
[    1.909232] Hardware name: Marvell Armada 380/385 (Device Tree)
[    1.915202] [<c010edac>] (unwind_backtrace) from [<c010aa74>] (show_stack+0x10/0x14)
[    1.922993] [<c010aa74>] (show_stack) from [<c0583cb4>] (dump_stack+0x94/0xa8)
[    1.930256] [<c0583cb4>] (dump_stack) from [<c010dbec>] (handle_IPI+0xe4/0x190)
[    1.937605] [<c010dbec>] (handle_IPI) from [<c0101494>] (gic_handle_irq+0x8c/0x90)
[    1.945214] [<c0101494>] (gic_handle_irq) from [<c010b70c>] (__irq_svc+0x6c/0x90)
[    1.952735] Exception stack(0xdf463f80 to 0xdf463fc8)
[    1.957814] 3f80: 00000001 00000000 00000000 c0114580 ffffe000 c0903c74 c0903c28 00000000
[    1.966035] 3fa0: 00000000 414fc091 00000000 00000000 df463fc8 df463fd0 c0108168 c010816c
[    1.974253] 3fc0: 60000013 ffffffff
[    1.977763] [<c010b70c>] (__irq_svc) from [<c010816c>] (arch_cpu_idle+0x34/0x38)
[    1.985200] [<c010816c>] (arch_cpu_idle) from [<c0154a98>] (do_idle+0xdc/0x19c)
[    1.992549] [<c0154a98>] (do_idle) from [<c0154db4>] (cpu_startup_entry+0x18/0x1c)
[    2.000159] [<c0154db4>] (cpu_startup_entry) from [<0010182c>] (0x10182c)
[    2.007420] Rebooting in 1 seconds..

This seems that the kernel failed to mount rootfs.
What's wrong?

I'm not aware of any openwrt support for QNAP devices. If your device is using QNAP's own firmware then you really should be contacting them for support.

3 Likes

This probably would have been simple to debridk, if you hadn't erased the backup partitions.

Especially with NAND flash, using the bootloader CLI to write directly to flash is difficult to get the OS Into flash properly, and a risk of accidentally erasing something important. Possibly the bootloader has an automatic debricking process like TP-Link or Ubiquity's TFTP based method or Netgear's nrmp. The best way to (manually) debrick is to boot an intiramfs image in RAM and use it to install the sysupgrade into flash. Like @krazeh said you should contact the manufacturer for support.

1 Like

Hi all,

Thank you for your advice.

I have 1 more QNAP QSW-M1208-8C.
In addition, I found USB Type-A port under the hood.
Thus, I have successfully retrieved and written kernel and rootfs images from the other one to the faulty one like below.

At first, I made ext4-formatted USB storage on Linux-PC.
Then, I retrieved kernel and rootfs images from the other one by the following commands in serial console.

# set USB-type as 3
setenv usbType 3
# enable USB
usb start
# check usb storage device number and partition number
usb part
# load kernel image from NAND
nand read 0x2000000 0xa80000 0xde00000
# write kernel image to USB storage
ext4write usb 0:1 /kernel1.img 0x2000000 0xde00000
# load rootfs image from NAND
nand read 0x2000000 0x1080000 0xd800000
# write rootfs image to USB storage
ext4write usb 0:1 /rootfs1.img 0x2000000 0xd800000
# load kernel image from NAND
nand read 0x2000000 0xe880000 0xde00000
# write kernel image to USB storage
ext4write usb 0:1 /kernel2.img 0x2000000 0xde00000
# load rootfs image from NAND
nand read 0x2000000 0xee80000 0xd800000
# write rootfs image to USB storage
ext4write usb 0:1 /rootfs2.img 0x2000000 0xd800000
# disable USB
usb stop

Finally, I wrote kernel and rootfs images to the faulty one by the following commands.

# set USB-type as 3
setenv usbType 3
# enable USB
usb start
# check usb storage device number and partition number
usb part
# load kernel image from USB storage
ext4load usb 0:1 0x2000000 /kernel1.img 0xde00000
# erase kernel1 partition in NAND
nand erase.part kernel1
# write kernel image to NAND
nand write 0x2000000 0xa80000 0xde00000
# load rootfs image from USB storage
ext4load usb 0:1 0x2000000 /rootfs2.img 0xd800000
# erase rootfs1 partition in NAND
nand erase.part rootfs1
# write rootfs image to NAND
nand write 0x2000000 0x1080000 0xd800000
# load kernel image from USB storage
ext4load usb 0:1 0x2000000 /kernel2.img 0xde00000
# erase kernel2 partition
nand erase.part kernel2
# write kernel image to NAND
nand write 0x2000000 0xe880000 0xde00000
# load rootfs image from USB storage
ext4load usb 0:1 0x2000000 /rootfs2.img 0xd800000
# erase rootfs2 partition
nand erase.part rootfs2
# write rootfs image to NAND
nand write 0x2000000 0xee80000 0xd800000

Note that rootfs1.img caused CRC error in booting and rootfs2.img is used for rootfs2 instead.
The file rootfs1.img might be broken for any reason.
Some environment variables of the faulty one might need to be changed to the same as the other one.

Best regards,

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.