[Solved] Sysupgrade to recent snapshot hangs on boot with "Waiting for root device /dev/ubiblock0_0"

Installing and Using OpenWrt
1

I have a Linksys EA8300 that I installed a snapshot build (circa 6/2) on using the factory firmware's update mechanism.

Two days ago, I tried upgrading to the latest snapshot to fix a problem with WDS client connections that has recently been patched. It never rebooted, and after three attempts the uboot failsafe kicked in, wiped the "alt" partition(s) and rebooted to the older snapshot I'd originally installed.

Yesterday I tried again with a day-newer snapshot. Same behavior. So, I opened up the case, soldered in some headers to get access to the serial console and tried again a couple of times.

Both times, things ended up here:

Waiting for root device /dev/ubiblock0_0
kernel messages 
[    0.906582] nand: 256 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[    0.910203] 16 fixed-partitions partitions found on MTD device qcom_nand.0
[    0.917757] Creating 16 MTD partitions on "qcom_nand.0":
[    0.924618] 0x000000000000-0x000000100000 : "sbl1"
[    0.931526] 0x000000100000-0x000000200000 : "mibib"
[    0.935998] 0x000000200000-0x000000300000 : "qsee"
[    0.940751] 0x000000300000-0x000000380000 : "cdt"
[    0.945310] 0x000000380000-0x000000400000 : "appsblenv"
[    0.950007] 0x000000400000-0x000000480000 : "ART"
[    0.955095] 0x000000480000-0x000000680000 : "appsbl"
[    0.961101] 0x000000680000-0x000000700000 : "u_env"
[    0.964968] 0x000000700000-0x000000740000 : "s_env"
[    0.969345] 0x000000740000-0x000000780000 : "devinfo"
[    0.974261] 0x000000780000-0x000005f80000 : "kernel"
[    0.984986] random: fast init done
[    1.046471] 0x000000a80000-0x000005f80000 : "rootfs"
[    1.112045] mtd: device 11 (rootfs) set to be root filesystem
[    1.112337] mtdsplit: no squashfs found in "rootfs"
[    1.116787] 0x000005f80000-0x00000b780000 : "alt_kernel"
[    1.189237] 0x000006280000-0x00000b780000 : "alt_rootfs"
[    1.254935] 0x00000b780000-0x00000b880000 : "sysdiag"
[    1.256338] 0x00000b880000-0x00000ff00000 : "syscfg"
[    1.314114] libphy: ipq40xx_mdio: probed
[    1.344951] ESS reset ok!
[    1.377915] ESS reset ok!
[    1.578024] PHY 2 single test PSGMII issue happen!
[    1.781493] PHY2 test see issue!
[    1.813584] ESS reset ok!
[    1.878680] PHY 0 single test PSGMII issue happen!
[    2.203821] PHY0 test see issue!
[    2.243991] ESS reset ok!
[    2.671409] libphy: Fixed MDIO Bus: probed
[    2.672988] EDMA using MAC@ - using
[    2.673001] 5e:f3:2d:bc:89:be
[    2.771169] i2c /dev entries driver
[    2.801959] NET: Registered protocol family 10
[    2.803835] Segment Routing with IPv6
[    2.805402] NET: Registered protocol family 17
[    2.809091] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    2.813808] 8021q: 802.1Q VLAN Support v1.8
[    2.826705] Registering SWP/SWPB emulation handler
[    2.834629] ubi0: attaching mtd13
[    3.143775] random: crng init done
[    3.221566] ubi0: scanning is finished
[    3.228741] ubi0: attached mtd13 (name "alt_rootfs", size 85 MiB)
[    3.228774] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    3.233836] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    3.240587] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    3.247467] ubi0: good PEBs: 680, bad PEBs: 0, corrupted PEBs: 0
[    3.254232] ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128
[    3.260471] ubi0: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1426662539
[    3.267513] ubi0: available PEBs: 0, total reserved PEBs: 680, PEBs reserved for bad PEB handling: 40
[    3.276816] ubi0: background thread "ubi_bgt0d" started, PID 85Ëť
[    3.292855] Waiting for root device /dev/ubiblock0_0...

It sits waiting for the root device indefinitely.

What can I do to get this working?

Sorry I don't have the full log, I had a "technical difficulty." If this doesn't have enough info I'll hook things up again and get it all.

Thanks!

1 Like
2

Enough to show the problem clearly, thanks. If you have a chance, knowing your “last good” build version number would help me try to replicate.

1 Like
3

I just flashed last night's snapshot, downloaded from http://downloads.openwrt.org/snapshots/targets/ipq40xx/generic/openwrt-ipq40xx-linksys_ea8300-squashfs-sysupgrade.bin and it boots cleanly for me.

 OpenWrt SNAPSHOT, r10206-35a70d6

and I was able to confirm as well that it had switched from the previous /dev/mtd13 for root and the ROM seems to be from last night's build:

jeff@deb-devel:~/devel/openwrt$ ./scripts/getver.sh 
r10206-35a70d6262
jeff@deb-devel:~/devel/openwrt$ git log -1 --pretty='%cd %h %s' --date=short 
2019-06-12 35a70d6262 f2fs-tools: fixup SPDX license
4

The version running on my ea8300 is: "OpenWrt SNAPSHOT r10121-06e63aa"

I just tried again (twice) using a newly downloaded file:

linksys_ea8300-squashfs-sysupgrade.bin
09a223e78649c634783478195b89a97d121b0b4b412b64d6f1988ee28a592e61 
5170.8 KB Thu Jun 13 02:51:23 2019

Same result, both times.

Expand for Console Log 
Watchdog handover: fd=3
- watchdog -
killall: telnetd: no process killed
killall: ash: no process killed
Sending TERM to remaining processes  ntpd wpa_supplicant hostapd ubusd askfirst dnsmasq logd rpcd netifd odhcpd uhttpd 
[  197.755345] ath10k_ahb a000000.wifi: peer-unmap-event: unknown peer id 0
[  197.755956] ath10k_ahb a000000.wifi: peer-unmap-event: unknown peer id 0
Sending KILL to remaining processes  
Switching to ramdisk
[  204.294758] UBIFS (ubi0:1): background thread ubifs_bgt0_1 stops
[  204.389805] UBIFS (ubi0:1): un-mount UBI device 0
Performing system upgrade
[  204.622335] ubi1: attaching mtd13
[  205.008263] ubi1: scanning is finished
[  205.015462] ubi1: attached mtd13 (name alt_rootfs, size 85 MiB)
[  205.015494] ubi1: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[  205.020537] ubi1: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[  205.027328] ubi1: VID header offset: 2048 (aligned 2048), data offset: 4096
[  205.034185] ubi1: good PEBs: 680, bad PEBs: 0, corrupted PEBs: 0
[  205.040938] ubi1: user volume: 1, internal volumes: 1, max. volumes count: 128
[  205.047209] ubi1: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1426662539
[  205.054234] ubi1: available PEBs: 0, total reserved PEBs: 680, PEBs reserved for bad PEB handling: 40
[  205.063538] ubi1: background thread ubi_bgt1d started, PID 2483
UBI device number 1, total 680 LEBs (86343680 bytes, 82.3 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)
Unlocking alt_kernel 

Writing from <stdin> to alt_kernel   [ ][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w]    
ubimkvol: error!: UBI device does not have free logical eraseblocks
cannot create rootfs volume
libubi: error!: /dev/ is not a character device
ubiupdatevol: error!: error while probing /dev/
              error 22 (Invalid argument)
tar: write error: Broken pipe
sysupgrade successful
umount: cant unmount /dev: Resource busy
umount: cant unmount /tmp: Resource busy
[
Format: Log Type - Time(microsec) - Message - Optional Info
Log Type: B - Since Boot(Power On Reset),  D - Delta,  S - Statistic
S - QC_IMAGE_VERSION_STRING=BOOT.BF.3.1.1-00108
S - IMAGE_VARIANT_STRING=DAACANAZA
S - OEM_IMAGE_VERSION_STRING=CRM
S - Boot Config, 0x00000025
S - Reset status Config, 0x00000010
S - Core 0 Frequency, 0 MHz
B -       261 - PBL, Start
B -      1339 - bootable_media_detect_entry, Start
B -      2612 - bootable_media_detect_success, Start
B -      2626 - elf_loader_entry, Start
B -      4035 - auth_hash_seg_entry, Start
B -      6190 - auth_hash_seg_exit, Start
B -     74322 - elf_segs_hash_verify_entry, Start
B -    196802 - PBL, End
B -    196826 - SBL1, Start
B -    288869 - pm_device_init, Start
D -         7 - pm_device_init, Delta
B -    290361 - boot_flash_init, Start
D -     87180 - boot_flash_init, Delta
B -    381847 - boot_config_data_table_init, Start
D -     13972 - boot_config_data_table_init, Delta - (419 Bytes)
B -    398582 - clock_init, Start
D -      7570 - clock_init, Delta
B -    409585 - CDT version:2,Platform ID:8,Major ID:1,Minor ID:0,Subtype:6
B -    412998 - sbl1_ddr_set_params, Start
B -    418093 - cpr_init, Start
D -         2 - cpr_init, Delta
B -    422475 - Pre_DDR_clock_init, Start
D -         4 - Pre_DDR_clock_init, Delta
D -     13171 - sbl1_ddr_set_params, Delta
B -    435780 - pm_driver_init, Start
D -         2 - pm_driver_init, Delta
B -    505627 - sbl1_wait_for_ddr_training, Start
D -        28 - sbl1_wait_for_ddr_training, Delta
B -    520955 - Image Load, Start
D -    143737 - QSEE Image Loaded, Delta - (269176 Bytes)
B -    665118 - Image Load, Start
D -      2118 - SEC Image Loaded, Delta - (2048 Bytes)
B -    675261 - Image Load, Start
D -    186943 - APPSBL Image Loaded, Delta - (444263 Bytes)
B -    862600 - QSEE Execution, Start
D -        56 - QSEE Execution, Delta
B -    868827 - SBL1, End
D -    674001 - SBL1, Delta
S - Flash Throughput, 2090 KB/s  (715906 Bytes,  342509 us)
S - DDR Frequency, 672 MHz


U-Boot 2012.07 [Chaos Calmer 15.05.1,r35193] (Nov 02 2017 - 16:33:09)

CBT U-Boot ver: 1.2.9

smem ram ptable found: ver: 1 len: 3
DRAM:  256 MiB
machid : 0x8010006
NAND:  ID = 9590dac2
Vendor = c2
Device = da
ONFI device found
SF NAND unsupported id:ff:ff:ff:ffSF: Unsupported manufacturer ff
ipq_spi: SPI Flash not found (bus/cs/speed/mode) = (0/0/48000000/0)
256 MiB
MMC:   qca_mmc: 0
PCI0 Link Intialized
In:    serial
Out:   serial
Err:   serial
machid: 8010006
flash_type: 2
Net:   MAC0 addr:0:3:7f:ba:db:ad
PHY ID1: 0x4d
PHY ID2: 0xd0b1
ipq40xx_ess_sw_init done
eth0

Updating boot_count  done

Hit any key to stop autoboot:  2  1  0 

Memory test start,  banksize=0x2000000, data size=0x8000, test count per bank=64
Memory test end, lapsed time = 3 seconds

The test result is success


NAND read: device 0 offset 0x5f80000, size 0x300000
 3145728 bytes read: OK
## Booting kernel from FIT Image at 84000000 
   Using config@1 configuration
   Trying kernel@1 kernel subimage
     Description:  ARM OpenWrt Linux-4.14.125
     Type:         Kernel Image
     Compression:  uncompressed
     Data Start:   0x840000e4
     Data Size:    2272176 Bytes = 2.2 MiB
     Architecture: ARM
     OS:           Linux
     Load Address: 0x80208000
     Entry Point:  0x80208000
     Hash algo:    crc32
     Hash value:   0320997a
     Hash algo:    sha1
     Hash value:   c7a18057b1da036118ce490eb56ce33f395a831b
   Verifying Hash Integrity  crc32+ sha1+ OK
## Flattened Device Tree from FIT Image at 84000000
   Using config@1 configuration
   Trying fdt@1 FDT blob subimage
     Description:  ARM OpenWrt linksys_ea8300 device tree blob
     Type:         Flat Device Tree
     Compression:  uncompressed
     Data Start:   0x8422adcc
     Data Size:    17347 Bytes = 16.9 KiB
     Architecture: ARM
     Hash algo:    crc32
     Hash value:   e828b72b
     Hash algo:    sha1
     Hash value:   eb01db16b1f68c863b28c8cb2d373703c8535a0a
   Verifying Hash Integrity  crc32+ sha1+ OK
   Booting using the fdt blob at 0x8422adcc
   Loading Kernel Image  OK
OK
   Loading Device Tree to 871e8000, end 871ef3c2  OK
Device nand2 not found!
eth0 MAC Address from ART is not valid
eth1 MAC Address from ART is not valid
Using machid 0x8010006 from environment

Starting kernel 

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 4.14.125 (buildbot@slashdirt-02) (gcc version 7.4.0 (OpenWrt GCC 7.4.0 r10206-35a70d6)) #0 SMP Wed Jun 12 21:55:53 2019
[    0.000000] CPU: ARMv7 Processor [410fc075] revision 5 (ARMv7), cr=10c5387d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
[    0.000000] OF: fdt: Machine model: Linksys EA8300 (Dallas)
[    0.000000] Memory policy: Data cache writealloc
[    0.000000] random: get_random_bytes called from start_kernel+0x8c/0x3ec with crng_init=0
[    0.000000] percpu: Embedded 15 pages/cpu s29260 r8192 d23988 u61440
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 64448
[    0.000000] Kernel command line: init=/sbin/init rootfstype=ubifs ubi.mtd=13,2048 root=ubi0:ubifs rootwait rw root=/dev/ubiblock0_0 rootfstype=squashfs ro
[    0.000000] PID hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000000] Dentry cache hash table entries: 32768 (order: 5, 131072 bytes)
[    0.000000] Inode-cache hash table entries: 16384 (order: 4, 65536 bytes)
[    0.000000] Memory: 249652K/260096K available (4647K kernel code, 163K rwdata, 1236K rodata, 1024K init, 241K bss, 10444K reserved, 0K cma-reserved, 0K highmem)
[    0.000000] Virtual kernel memory layout:
[    0.000000]     vector  : 0xffff0000 - 0xffff1000   (   4 kB)
[    0.000000]     fixmap  : 0xffc00000 - 0xfff00000   (3072 kB)
[    0.000000]     vmalloc : 0xd0800000 - 0xff800000   ( 752 MB)
[    0.000000]     lowmem  : 0xc0000000 - 0xd0000000   ( 256 MB)
[    0.000000]     pkmap   : 0xbfe00000 - 0xc0000000   (   2 MB)
[    0.000000]     modules : 0xbf000000 - 0xbfe00000   (  14 MB)
[    0.000000]       .text : 0xc0208000 - 0xc0789ce8   (5640 kB)
[    0.000000]       .init : 0xc0900000 - 0xc0a00000   (1024 kB)
[    0.000000]       .data : 0xc0a00000 - 0xc0a28d00   ( 164 kB)
[    0.000000]        .bss : 0xc0a2a000 - 0xc0a66458   ( 242 kB)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
[    0.000000] arch_timer: cp15 timer(s) running at 48.00MHz (virt).
[    0.000000] clocksource: arch_sys_counter: mask: 0xffffffffffffff max_cycles: 0xb11fd3bfb, max_idle_ns: 440795203732 ns
[    0.000009] sched_clock: 56 bits at 48MHz, resolution 20ns, wraps every 4398046511096ns
[    0.000022] Switching to timer-based delay loop, resolution 20ns
[    0.000255] Calibrating delay loop (skipped), value calculated using timer frequency.. 96.00 BogoMIPS (lpj=480000)
[    0.000274] pid_max: default: 32768 minimum: 301
[    0.000453] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.000470] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes)
[    0.001221] CPU: Testing write buffer coherency: ok
[    0.001964] Setting up static identity map for 0x80300000 - 0x80300060
[    0.002123] Hierarchical SRCU implementation.
[    0.002803] smp: Bringing up secondary CPUs 
[    0.005589] smp: Brought up 1 node, 4 CPUs
[    0.005610] SMP: Total of 4 processors activated (384.00 BogoMIPS).
[    0.005619] CPU: All CPU(s) started in SVC mode.
[    0.010202] VFP support v0.3: implementor 41 architecture 2 part 30 variant 7 rev 5
[    0.010365] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.010388] futex hash table entries: 1024 (order: 4, 65536 bytes)
[    0.010642] pinctrl core: initialized pinctrl subsystem
[    0.011558] NET: Registered protocol family 16
[    0.011895] DMA: preallocated 256 KiB pool for atomic coherent allocations
[    0.012896] cpuidle: using governor ladder
[    0.012941] cpuidle: using governor menu
[    0.030491] usbcore: registered new interface driver usbfs
[    0.030553] usbcore: registered new interface driver hub
[    0.030632] usbcore: registered new device driver usb
[    0.030678] pps_core: LinuxPPS API ver. 1 registered
[    0.030687] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.030725] PTP clock support registered
[    0.031963] clocksource: Switched to clocksource arch_sys_counter
[    0.032841] NET: Registered protocol family 2
[    0.033514] TCP established hash table entries: 2048 (order: 1, 8192 bytes)
[    0.033558] TCP bind hash table entries: 2048 (order: 2, 16384 bytes)
[    0.033608] TCP: Hash tables configured (established 2048 bind 2048)
[    0.033731] UDP hash table entries: 256 (order: 1, 8192 bytes)
[    0.033765] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes)
[    0.033983] NET: Registered protocol family 1
[    0.035097] No memory allocated for crashlog
[    0.035288] workingset: timestamp_bits=14 max_order=16 bucket_order=2
[    0.039198] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    0.039215] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    0.044871] io scheduler noop registered
[    0.044890] io scheduler deadline registered (default)
[    0.046652] OF: PCI: host bridge /soc/pci@40000000 ranges:
[    0.046688] OF: PCI:    IO 0x40200000..0x402fffff -> 0x40200000
[    0.046706] OF: PCI:   MEM 0x40300000..0x40ffffff -> 0x40300000
[    0.261970] qcom-pcie 40000000.pci: link up
[    0.262121] qcom-pcie 40000000.pci: PCI host bridge to bus 0000:00
[    0.262143] pci_bus 0000:00: root bus resource [bus 00-ff]
[    0.262160] pci_bus 0000:00: root bus resource [io  0x0000-0xfffff] (bus address [0x40200000-0x402fffff])
[    0.262173] pci_bus 0000:00: root bus resource [mem 0x40300000-0x40ffffff]
[    0.262502] PCI: bus0: Fast back to back transfers disabled
[    0.263869] PCI: bus1: Fast back to back transfers disabled
[    0.263923] pci 0000:00:00.0: BAR 8: assigned [mem 0x40400000-0x405fffff]
[    0.263939] pci 0000:00:00.0: BAR 0: assigned [mem 0x40300000-0x40300fff 64bit]
[    0.263964] pci 0000:01:00.0: BAR 0: assigned [mem 0x40400000-0x405fffff 64bit]
[    0.264040] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    0.264058] pci 0000:00:00.0:   bridge window [mem 0x40400000-0x405fffff]
[    0.264823] pcieport 0000:00:00.0: AER enabled with IRQ 93
[    0.266270] bam-dma-engine 8e04000.dma: num-channels unspecified in dt
[    0.266286] bam-dma-engine 8e04000.dma: num-ees unspecified in dt
[    0.267508] tcsr 1949000.tcsr: setting wifi_glb_cfg = 41000000
[    0.267588] tcsr 194b000.tcsr: setting usb hs phy mode select = e700e7
[    0.267663] tcsr 1953000.ess_tcsr: setting ess interface select = 0
[    0.267736] tcsr 1957000.tcsr: setting wifi_noc_memtype_m0_m2 = 2222222
[    0.267965] Serial: 8250/16550 driver, 16 ports, IRQ sharing enabled
[    0.269684] msm_serial 78af000.serial: msm_serial: detected port #0
[    0.269733] msm_serial 78af000.serial: uartclk = 1843200
[    0.269787] 78af000.serial: ttyMSM0 at MMIO 0x78af000 (irq = 24, base_baud = 115200) is a MSM
[    0.269813] msm_serial: console setup on port #0
[    0.885550] console [ttyMSM0] enabled
[    0.890443] msm_serial: driver initialized
[    0.898251] loop: module loaded
[    0.899749] nand: device found, Manufacturer ID: 0xc2, Chip ID: 0xda
[    0.900792] nand: Macronix MX30LF2G18AC
[    0.907434] nand: 256 MiB, SLC, erase size: 128 KiB, page size: 2048, OOB size: 64
[    0.911041] 16 fixed-partitions partitions found on MTD device qcom_nand.0
[    0.918609] Creating 16 MTD partitions on qcom_nand.0:
[    0.925470] 0x000000000000-0x000000100000 : sbl1
[    0.932410] 0x000000100000-0x000000200000 : mibib
[    0.936899] 0x000000200000-0x000000300000 : qsee
[    0.941634] 0x000000300000-0x000000380000 : cdt
[    0.946183] 0x000000380000-0x000000400000 : appsblenv
[    0.950928] 0x000000400000-0x000000480000 : ART
[    0.956009] 0x000000480000-0x000000680000 : appsbl
[    0.961931] 0x000000680000-0x000000700000 : u_env
[    0.965913] 0x000000700000-0x000000740000 : s_env
[    0.970241] 0x000000740000-0x000000780000 : devinfo
[    0.975168] 0x000000780000-0x000005f80000 : kernel
[    0.985764] random: fast init done
[    1.047235] 0x000000a80000-0x000005f80000 : rootfs
[    1.112634] mtd: device 11 (rootfs) set to be root filesystem
[    1.112935] mtdsplit: no squashfs found in rootfs
[    1.117376] 0x000005f80000-0x00000b780000 : alt_kernel
[    1.189755] 0x000006280000-0x00000b780000 : alt_rootfs
[    1.255278] 0x00000b780000-0x00000b880000 : sysdiag
[    1.256703] 0x00000b880000-0x00000ff00000 : syscfg
[    1.314518] libphy: ipq40xx_mdio: probed
[    1.345454] ESS reset ok!
[    1.378422] ESS reset ok!
[    1.812343] libphy: Fixed MDIO Bus: probed
[    1.813939] EDMA using MAC@ - using
[    1.813953] 1e:7c:1d:7c:df:34
[    1.912109] i2c /dev entries driver
[    1.943075] NET: Registered protocol family 10
[    1.944544] Segment Routing with IPv6
[    1.946487] NET: Registered protocol family 17
[    1.950190] bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
[    1.954846] 8021q: 802.1Q VLAN Support v1.8
[    1.967771] Registering SWP/SWPB emulation handler
[    1.975805] ubi0: attaching mtd13
[    2.305694] random: crng init done
[    2.386923] ubi0: scanning is finished
[    2.394384] ubi0: attached mtd13 (name alt_rootfs, size 85 MiB)
[    2.394416] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[    2.399458] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[    2.406263] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
[    2.413102] ubi0: good PEBs: 680, bad PEBs: 0, corrupted PEBs: 0
[    2.419858] ubi0: user volume: 1, internal volumes: 1, max. volumes count: 128
[    2.426124] ubi0: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1426662539
[    2.433166] ubi0: available PEBs: 0, total reserved PEBs: 680, PEBs reserved for bad PEB handling: 40
[    2.442463] ubi0: background thread ubi_bgt0d started, PID 86
[    2.458518] Waiting for root device /dev/ubiblock0_0

This time I caught the console output during sysupgrade (excerpt below). It looks like this may be where the problem starts...

Watchdog handover: fd=3
- watchdog -
killall: telnetd: no process killed
killall: ash: no process killed
Sending TERM to remaining processes  ntpd wpa_supplicant hostapd ubusd askfirst dnsmasq logd rpcd netifd odhcpd uhttpd 
[  197.755345] ath10k_ahb a000000.wifi: peer-unmap-event: unknown peer id 0
[  197.755956] ath10k_ahb a000000.wifi: peer-unmap-event: unknown peer id 0
Sending KILL to remaining processes  
Switching to ramdisk
[  204.294758] UBIFS (ubi0:1): background thread ubifs_bgt0_1 stops
[  204.389805] UBIFS (ubi0:1): un-mount UBI device 0
Performing system upgrade
[  204.622335] ubi1: attaching mtd13
[  205.008263] ubi1: scanning is finished
[  205.015462] ubi1: attached mtd13 (name alt_rootfs, size 85 MiB)
[  205.015494] ubi1: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
[  205.020537] ubi1: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
[  205.027328] ubi1: VID header offset: 2048 (aligned 2048), data offset: 4096
[  205.034185] ubi1: good PEBs: 680, bad PEBs: 0, corrupted PEBs: 0
[  205.040938] ubi1: user volume: 1, internal volumes: 1, max. volumes count: 128
[  205.047209] ubi1: max/mean erase counter: 2/0, WL threshold: 4096, image sequence number: 1426662539
[  205.054234] ubi1: available PEBs: 0, total reserved PEBs: 680, PEBs reserved for bad PEB handling: 40
[  205.063538] ubi1: background thread ubi_bgt1d started, PID 2483
UBI device number 1, total 680 LEBs (86343680 bytes, 82.3 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)
Unlocking alt_kernel 

Writing from <stdin> to alt_kernel   [ ][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w][e][w]    
ubimkvol: error!: UBI device does not have free logical eraseblocks
cannot create rootfs volume
libubi: error!: /dev/ is not a character device
ubiupdatevol: error!: error while probing /dev/
              error 22 (Invalid argument)
tar: write error: Broken pipe
sysupgrade successful
umount: cant unmount /dev: Resource busy
umount: cant unmount /tmp: Resource busy
5

Contrary to “upgrade successful”, it didn’t write a root file system, hence why the kernel can’t find it on boot.

To confirm, as far as you know, it is trying to overwrite OEM firmware from your first-time flash of OpenWrt, correct?

Edit

Still haven’t had my coffee here, but it looks like sysupgrade on the running device is failing, never finding the target volume to flash. /dev/ should be /dev/ubi1_1 or the like. Based on that, newer downloads are unlikely to fix the problem.

Building off 06e63aa now to see if I can replicate.

Edit:

Successfully flashes from 06e63aa

Just to confirm, you've got a functional router at this time, though you'd prefer the newer firmware, and you have serial access (for U-Boot), correct?

6

I just tried to do sysupgrade from today’s file. Is there a way to revert to older firmare or stock without serial access?.
Also ssh is getting connection refused.

7

@syto203 Would you describe the problem you’re having, if any. Sometimes what seems to be “the same problem” turns out to be something different.

I should be able to provide steps for that, I just need to know the state of your EA8300.

8

I triggered failsafe and reverted, tried reconnecting and it failed but it seems the problem was something local since renewing my connections fixed it.

My current build is r10199-04b45d3a31.
does current sysupgrade have issues or was it just a fluke.

Now I’m looking for a way to revert to stock, do I just flash the stock firmware, asking since you said the device already juggles between both firmwares.

9

I don't yet know why there is an issue. I haven't been able to replicate it locally.

OEM firmware cannot be directly flashed by sysupgrade and, if there is some problem with sysupgrade, that wouldn't be a reliable option.

Right now, my priority is figuring out why there was a sysupgrade failure. Determining how to robustly guide a non-expert in flashing OEM firmware from running OpenWrt is lower priority.

Edit: How many times have you flashed OpenWrt? If only once, you should still have OEM firmware as the "other" firmware. You should be able to switch back to that with luci-app-advanced-reboot, or by examining the current value returned by fw_printenv boot_part and switching to the other firmware with fw_setenv boot_part N where N is 2 if it is presently 1, or is 1 if presently 2.

1 Like
10

Ok good luck.

Edit: thanks for the extra info.
Advanced Reboot reports that "linksys ea8300 is unknown or isn't a dual partition device".
fw_printenv reports boot_part= 2.

11

@eas -- If you have time, would you be able to run the script from Script: Mount "Alternate" NAND Firmware (Linksys +?) -- I called it mount_alt.sh on my machine.

sh -vx ./mount_alt.sh

will echo the line, as well as the interpolations. I'm looking for errors to understand what might be wrong. From my EA8300, it looked like

root@test:~# sh -vx ./mount_alt.sh 
#!/bin/sh

if [ "$( cat /sys/devices/virtual/ubi/ubi0/mtd_num )" = "11" ] ; then
	altpart=13
else
	altpart=11
fi
+ cat /sys/devices/virtual/ubi/ubi0/mtd_num
+ '[' 11 '=' 11 ]
+ altpart=13

mkdir -p /alt/rom
+ mkdir -p /alt/rom
mkdir -p /alt/overlay
+ mkdir -p /alt/overlay
mkdir -p /alt/firmware
+ mkdir -p /alt/firmware

ubiattach -m ${altpart}
+ ubiattach -m 13
UBI device number 1, total 680 LEBs (86343680 bytes, 82.3 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)

ubiblock --create /dev/ubi1_0
+ ubiblock --create /dev/ubi1_0
mount -t squashfs -o ro /dev/ubiblock1_0 /alt/rom
+ mount -t squashfs -o ro /dev/ubiblock1_0 /alt/rom

mount -t ubifs /dev/ubi1_1 /alt/overlay
+ mount -t ubifs /dev/ubi1_1 /alt/overlay

mount -t overlay overlay -o noatime,lowerdir=/alt/rom,upperdir=/alt/overlay/upper,workdir=/alt/overlay/work /alt/firmware
+ mount -t overlay overlay -o 'noatime,lowerdir=/alt/rom,upperdir=/alt/overlay/upper,workdir=/alt/overlay/work' /alt/firmware
root@test:~#
1 Like
12

Yes, my router boots into the OpenWRT snapshot I installed around the beginning of June (OpenWrt SNAPSHOT r10121-06e63aa). The WDS client mode is flakey, as described elsewhere, and I want to confirm that some kernel patches applied to snapshot at the beginning of the week will fix the problem.

I do have serial access to the console, which is how I'm getting the logs I've posted. I haven't tried intervening with UBoot, yet, but the option is there.

As far as I know, the OEM firmware is at least partially gone, since after sysupgrade, it looks like a boots a newer kernel build (log shows June 12) before choking on trying to mount the non-existant new root fs.

Reiterating, I don't actually know what happened the first time it failed since I hadn't opened the case and connected a serial cable.

Where should I/we go from here?

I'm willing to try to gather more info to try and help figure out what went wrong in the first place, in case there is a problem that could trip up others in the future (I guess, at the very least, sysupgrade isn't handling exceptions properly).

If we've exhausted that avenue, I'd appreciate help getting back to a system that can be upgraded normally.

Thanks for working through this with me.

Update: missed this:

Yeah, I'll do it tomorrow (Saturday). I had to pack the router up today to make room for repairing the 2.4GHz WiFi ( and GPS and Bluetooth) on my wife's iPhone. Very tedious getting in deep enough to replace the suspect parts. It would have been a real bummer if it didn't fix the issue, but signs are good. I'll pull the EA8300 out tomorrow.

1 Like
13

OK, I think I literally just replicated it here, or at least parts of it.

Yep

[    2.448095] ubi0: background thread "ubi_bgt0d" started, PID 85ďż˝[    2.464144] Waiting for root device /dev/ubiblock0_0...

Problem seems to be flashing over OEM -- at least I can replicate it here now.

Now that I can replicate it, please feel free to "fix up" your other partition.

I'd flash an (OpenWrt) factory.img using U-Boot and TFTP as the easiest way.

Which was last slated to boot (and will be next time)

boot_part=2

TFTP parameters; IP addresses can be changed just by setenv -- no save required, and will revert on boot if not saved (I only change them for "right now" and don't save)

image=dallas.img
ipaddr=192.168.1.1
netmask=255.255.255.0
serverip=192.168.1.254

then

run flashimg

or

run flashimg2

depending on the one you want to flash.

If that's not enough detail, please let me know.

Edit:

You could, for example, if running off firmware 1:

  • Flash firmware 2 with OpenWrt
  • Boot existing, configured firmware 1
  • Sysupgrade to firmware 2, retaining your current settings
1 Like
14

I've found the problem and it can be resolved prior to me deciding the best way to "fix" the upstream source.

What happens is that the OEM firmware has already taken over the root file system partition in a way that sysupgrade, right now, can't take it back. It can be resolved "by hand" using the command line (over SSH) relatively easily.

Step 1) Determine which partition you've booted into

root@test:~# cat /sys/devices/virtual/ubi/ubi0/mtd_num 
11

:warning: If this is 11, then you will be working with 13 in the later steps, if it is 13, then use 11

Step 2) Connect the "other" firmware to a UBI device

root@test:~# ubiattach -m 13
UBI device number 1, total 680 LEBs (86343680 bytes, 82.3 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)

If this does not say UBI device number 1 here, stop now and ask for clarification.

Step 3) Confirm that there is an OEM volume there that needs to be removed

root@test:~# ubinfo -d 1 -a
ubi1
Volumes count:                           1
Logical eraseblock size:                 126976 bytes, 124.0 KiB
Total amount of logical eraseblocks:     680 (86343680 bytes, 82.3 MiB)
Amount of available logical eraseblocks: 0 (0 bytes)
Maximum count of volumes                 128
Count of bad physical eraseblocks:       0
Count of reserved physical eraseblocks:  40
Current maximum erase counter value:     1
Minimum input/output unit size:          2048 bytes
Character device major/minor:            246:0
Present volumes:                         0

Volume ID:   0 (on ubi1)
Type:        dynamic
Alignment:   1
Size:        636 LEBs (80756736 bytes, 77.0 MiB)
State:       OK
Name:        ubifs
Character device major/minor: 246:1

If is says Name: ubifs then proceed. If not, recheck your steps, especially which partition you've booted into, and don't proceed if you aren't seeing Name: ubifs

Step 4) Remove the OEM ubifs volume

root@test:~# ubirmvol /dev/ubi1 -N ubifs

You should be able to use sysupgrade now.

Patch submitted http://patchwork.ozlabs.org/patch/1116472/

Downloading https://gist.github.com/jeffsf/cd516fb84d0a46f1c3f9d49f2abea7ad and replacing /lib/upgrade/linksys.sh with it should give you the effect of that patch on your local, running install.

2 Likes
15

FWIW, the output when run on my ea8300:

Test Script Output 
#!/bin/sh

if [ "$( cat /sys/devices/virtual/ubi/ubi0/mtd_num )" = "11" ] ; then
	altpart=13
else
	altpart=11
fi
+ cat /sys/devices/virtual/ubi/ubi0/mtd_num
+ '[' 11 '=' 11 ]
+ altpart=13

mkdir -p /alt/rom
+ mkdir -p /alt/rom
mkdir -p /alt/overlay
+ mkdir -p /alt/overlay
mkdir -p /alt/firmware
+ mkdir -p /alt/firmware

ubiattach -m ${altpart}
+ ubiattach -m 13
UBI device number 1, total 680 LEBs (86343680 bytes, 82.3 MiB), available 0 LEBs (0 bytes), LEB size 126976 bytes (124.0 KiB)

ubiblock --create /dev/ubi1_0
+ ubiblock --create /dev/ubi1_0
mount -t squashfs -o ro /dev/ubiblock1_0 /alt/rom
+ mount -t squashfs -o ro /dev/ubiblock1_0 /alt/rom
mount: mounting /dev/ubiblock1_0 on /alt/rom failed: Invalid argument

mount -t ubifs /dev/ubi1_1 /alt/overlay
+ mount -t ubifs /dev/ubi1_1 /alt/overlay
mount: mounting /dev/ubi1_1 on /alt/overlay failed: Invalid argument

mount -t overlay overlay -o noatime,lowerdir=/alt/rom,upperdir=/alt/overlay/upper,workdir=/alt/overlay/work /alt/firmware
+ mount -t overlay overlay -o 'noatime,lowerdir=/alt/rom,upperdir=/alt/overlay/upper,workdir=/alt/overlay/work' /alt/firmware
mount: mounting overlay on /alt/firmware failed: No such file or directory

root@ea8300:~# cat /sys/devices/virtual/ubi/ubi0/mtd_num
11
root@ea8300:~# ubiattach -m 13
ubiattach: error!: cannot attach mtd13
           error 17 (File exists)
root@ea8300:~#

Stopped, awaiting clarification :smiley:
Update: Now I see that it was left attatched by your diagnostic script, and that the output looks right, so, I'll try sysupgrade tomorrow when I'm in front of the device and can capture the console output.

1 Like
16

And sysupgrade succeeded! Thanks for getting OpenWRT working on the EA8300 in the first place, and thanks for helping me get over this hump!

1 Like
17

@eas If your problem is solved, please consider marking this topic as [Solved]. (Click the pencil behind the topic...)

You can also mark the reply that solved your problem:
grafik

1 Like
18

so does this mean that the OEM firmware is already corrupt when i tried to sysupgrade and there is no need for it to remain OR is this the only way to be able to use sysupgrade?

however, if the OEM firmware is intact and i want it to remain an option without having to manually flash it, couldn't i just revert to it using fw_setenv and re-flash factory openwrt to get the latest fixes?

19

If you have run into this already, the other firmware is an OpenWrt kernel and an OEM root file system, which isn’t viable. Uploading a single file to the running OpenWrt is perhaps the easiest.

Flashing OpenWrt over itself, while possible, isn't something I’d generally recommend to a non-technical user. If they really wanted to potentially revert to OEM in the future, postpone that until you’ve got running firmware on both partitions and an actual need.

20

just to make sure I’ve understood this right.

  1. I’ve lost the ability to boot into OEM firmware since sysupgrade tried to replace it and half succeeded by only replacing the kernel and not the root partition.
  2. the only way to get back to stock would be via serial.
  3. if I’m booted in OEM should’nt flashing over the “other” would effectively overwrite it even if it was openwrt.

note: I have a fully functioning openwrt firmware.