Kernel(?) problems with RT3888-based Belkin F9K1109 (N750 DB) in 2024

After using the Padawan firmware for about 10 years(!) I have now installed OpenWRT on this router that most would not deem worthy today and that probably has very few users that update often.
However, it fulfills all my needs and I would very much like to continue using it.
There are two somewhat major problem I see with my self-built images based on a recent HEAD commit (66edb2f9ad):

  1. There is UART output in uboot and from the kernel but there are diagnostic messages regarding it and also no shell spawned after boot.
  2. Custom files from the image are not installed on upgrades for some reason while the writable overlay does seem to work in principle.

The log below shows the UART output starting at the point I issue the sysupgrade -n call from an SSH session to the completed reboot afterwards.

[  605.162138] rt2800pci 0000:01:00.0 phy0-ap0: left allmulticast mode
[  605.174597] rt2800pci 0000:01:00.0 phy0-ap0: left promiscuous mode
[  605.186922] br-lan: port 3(phy0-ap0) entered disabled state
[  605.899997] br-lan: port 2(phy1-ap0) entered disabled state
[  605.984302] rt2800_wmac 10180000.wmac phy1-ap0 (unregistering): left allmulticast mode
[  606.000058] rt2800_wmac 10180000.wmac phy1-ap0 (unregistering): left promiscuous mode
[  606.015551] br-lan: port 2(phy1-ap0) entered disabled state
[  620.135535] stage2 (3706): drop_caches: 3
[  628.636468] ------------[ cut here ]------------
[  628.645595] WARNING: CPU: 0 PID: 3706 at fs/super.c:695 0x8011e200
[  628.657825] VFS: Busy inodes after unmount of jffs2 (jffs2)
[  628.657839] Modules linked in: jitterentropy_rng rt2800soc(O) rt2800pci(O) rt2800mmio(O) rt2800lib(O) option nft_fib_inet nf_flow_table_inet usb_wwan rt2x00soc(O) rt2x00pci(O) rt2x00mmio(O) rt2x00lib(O) nft_reject_ipv6 nft_reject_ipv4 nft_reject_inet nft_reject nft_redir nft_quota nft_numgen nft_nat nft_masq nft_log nft_limit nft_hash nft_flow_offload nft_fib_ipv6 nft_fib_ipv4 nft_fib nft_ct nft_chain_nat nf_tables nf_nat nf_flow_table nf_conntrack mac80211(O) huawei_cdc_ncm cfg80211(O) cdc_ncm cdc_ether wwan usbserial usbnet nfnetlink nf_reject_ipv6 nf_reject_ipv4 nf_log_syslog nf_defrag_ipv6 nf_defrag_ipv4 libcrc32c crc_ccitt compat(O) cdc_wdm ledtrig_usbport nbd eeprom_93cx6 sha512_generic sha3_generic seqiv drbg hmac geniv rng cmac uas usb_storage leds_gpio ohci_platform ohci_hcd fsl_mph_dr_of ehci_platform ehci_fsl sd_mod scsi_mod scsi_common ehci_hcd loop gpio_button_hotplug(O) ext4 mbcache jbd2 usbcore nls_base usb_common crc16 mii crc32c_generic
[  628.837840] CPU: 0 PID: 3706 Comm: busybox Tainted: G        W  O       6.6.34 #0
[  628.852660] Stack : 804a5ffc 00000000 00000000 8004efb8 00000000 00000000 00000000 00000000
[  628.869273]         00000000 00000000 00000000 00000000 00000000 00000001 82d5bd38 1f603802
[  628.885888]         82d5bdd0 00000000 00000000 82d5bc78 00000038 8045c804 00000000 0000000c
[  628.902503]         20205720 80993360 00000001 2020204f 80473f0c 80473f0c 82d5be38 00000009
[  628.919118]         8047b4d4 77e7d3c8 00000000 7feed6f0 00000000 00000000 00000000 806d0000
[  628.935733]         ...
[  628.940580] Call Trace:
[  628.940592] [<8004efb8>] 0x8004efb8
[  628.952561] [<8045c804>] 0x8045c804
[  628.959534] [<8000606c>] 0x8000606c
[  628.966311] [<80006074>] 0x80006074
[  628.973232] [<8043da7c>] 0x8043da7c
[  628.980220] [<80021328>] 0x80021328
[  628.987142] [<8011e200>] 0x8011e200
[  628.994002] [<8011e200>] 0x8011e200
[  629.000924] [<800213d8>] 0x800213d8
[  629.008100] [<8011e200>] 0x8011e200
[  629.014876] [<8045f508>] 0x8045f508
[  629.021798] [<80114db8>] 0x80114db8
[  629.028969] [<802830e0>] 0x802830e0
[  629.035994] [<80195c2c>] 0x80195c2c
[  629.042833] [<8011f17c>] 0x8011f17c
[  629.050002] [<8013d49c>] 0x8013d49c
[  629.056843] [<8003971c>] 0x8003971c
[  629.063765] [<80001af0>] 0x80001af0
[  629.070644] 
[  629.073614] ---[ end trace 0000000000000000 ]---
[  629.185669] do_stage2 (3706): drop_caches: 3
[  630.309941] reboot: Restarting system


ARC Uboot:1.7.4 (Sep 14 2012 - 15:09:21)

Board: Ralink APSoC DRAM:  64 MB
relocate_code Pointer at: 83fa0000
******************************
Software System Reset Occurred
******************************
spi_wait_nsec: 30 
spi device id: c2 20 17 c2 20 (2017c220)
find flash: MX25L6405D
..============================================ 
Ralink UBoot Version: 3.5.2.0
-------------------------------------------- 
ASIC 3883_MP (MAC to VITESSE Mode)
DRAM_CONF_FROM: Boot-Strapping 
DRAM_TYPE: DDR2 
DRAM_SIZE: 512 Mbits
DRAM_WIDTH: 16 bits
DRAM_TOTAL_WIDTH: 16 bits
TOTAL_MEMORY_SIZE: 64 MBytes
Flash component: SPI Flash
Date:Sep 14 2012  Time:15:09:21
============================================ 
icache: sets:512, ways:4, linesz:32 ,total:65536
dcache: sets:256, ways:4, linesz:32 ,total:32768 

 ##### The CPU freq = 500 MHZ #### 
 estimate memory size =64 Mbytes

Please choose the operation: 
   2: Load system code then write to Flash via TFTP. 
   3: Boot system code via Flash (default).
   4: Entr boot command line interface.
   7: Load Boot Loader code then write to Flash via Serial. 
   9: Load Boot Loader code then write to Flash via TFTP. 
 4 initializing CHIP_RTL8367R_VB 1010
 0 
initializing CHIP_RTL8367R_VB 1010
   
3: System Boot system code via Flash.
## Booting image at bc050000 ...
.   Image Name:   N750F9K1103VB
   Created:      2024-06-26  23:53:43 UTC
   Image Type:   MIPS Linux Kernel Image (uncompressed)
   Data Size:    1835850 Bytes =  1.8 MB
   Load Address: 80000000
   Entry Point:  80000000
.............................   Verifying Checksum ... OK
OK
No initrd
## Transferring control to Linux (at address 80000000) ...
## Giving linux memsize in MB, 64

Starting kernel ...



OpenWrt kernel loader for MIPS based SoC
Copyright (C) 2011 Gabor Juhos <juhosg@openwrt.org>
Decompressing kernel... done!
Starting kernel at 80000000...

[    0.000000] Linux version 6.6.34 (stefanct@legacy) (mipsel-openwrt-linux-musl-gcc (OpenWrt GCC 13.3.0 r0+26740-66edb2f9ad) 13.3.0, GNU ld (GNU Binutils) 2.42) #0 Wed Jun 26 23:53:43 2024
[    0.000000] SoC Type: Ralink RT3883 ver:1 eco:5
[    0.000000] printk: bootconsole [early0] enabled
[    0.000000] CPU0 revision is: 0001974c (MIPS 74Kc)
[    0.000000] MIPS: machine is Belkin F9K1109 Version 1.0
[    0.000000] Initrd not found or empty - disabling initrd
[    0.000000] Primary instruction cache 64kB, VIPT, 4-way, linesize 32 bytes.
[    0.000000] Primary data cache 32kB, 4-way, VIPT, cache aliases, linesize 32 bytes
[    0.000000] Zone ranges:
[    0.000000]   Normal   [mem 0x0000000000000000-0x0000000003ffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000000000-0x0000000003ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000000000-0x0000000003ffffff]
[    0.000000] Kernel command line: console=ttyS0,57600 rootfstype=squashfs,jffs2
[    0.000000] Dentry cache hash table entries: 8192 (order: 3, 32768 bytes, linear)
[    0.000000] Inode-cache hash table entries: 4096 (order: 2, 16384 bytes, linear)
[    0.000000] Writing ErrCtl register=00000000
[    0.000000] Readback ErrCtl register=00000000
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 16256
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 57724K/65536K available (4491K kernel code, 576K rwdata, 720K rodata, 1180K init, 190K bss, 7812K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=32, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] NR_IRQS: 256
[    0.000000] ------------[ cut here ]------------
[    0.000000] WARNING: CPU: 0 PID: 0 at drivers/clk/ralink/clk-mtmips.c:499 0x80243f14
[    0.000000] Modules linked in:
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.6.34 #0
[    0.000000] Stack : 804a5ffc 00000000 00000004 8004efb8 00000000 00000000 00000000 00000000
[    0.000000]         00000000 00000000 00000000 00000000 00000000 00000001 8051bbf8 00000000
[    0.000000]         8051bc90 00000000 00000000 8051bb38 00000038 8045c804 00000000 00000000
[    0.000000]         203a6d6d 8051bb78 8051be48 70617773 80473f0c 80473f0c 00000000 00000009
[    0.000000]         00000000 00000001 00000004 00000001 00000000 00000000 83ffb780 00000000
[    0.000000]         ...
[    0.000000] Call Trace:
[    0.000000] [<8004efb8>] 0x8004efb8
[    0.000000] [<8045c804>] 0x8045c804
[    0.000000] [<8000606c>] 0x8000606c
[    0.000000] [<80006074>] 0x80006074
[    0.000000] [<8043da7c>] 0x8043da7c
[    0.000000] [<80021328>] 0x80021328
[    0.000000] [<80243f14>] 0x80243f14
[    0.000000] [<80243f14>] 0x80243f14
[    0.000000] [<800213d8>] 0x800213d8
[    0.000000] [<8045f6c4>] 0x8045f6c4
[    0.000000] [<80243f14>] 0x80243f14
[    0.000000] [<8023e0f8>] 0x8023e0f8
[    0.000000] [<8023e448>] 0x8023e448
[    0.000000] [<8023e750>] 0x8023e750
[    0.000000] [<805bb668>] 0x805bb668
[    0.000000] [<805bb6a0>] 0x805bb6a0
[    0.000000] [<802c30e8>] 0x802c30e8
[    0.000000] [<802c3194>] 0x802c3194
[    0.000000] [<805bb3b8>] 0x805bb3b8
[    0.000000] [<805ac348>] 0x805ac348
[    0.000000] [<805b2020>] 0x805b2020
[    0.000000] [<805adb58>] 0x805adb58
[    0.000000] [<805a9acc>] 0x805a9acc
[    0.000000] [<805a92f8>] 0x805a92f8
[    0.000000] 
[    0.000000] ---[ end trace 0000000000000000 ]---
[    0.000000] CPU Clock: 500MHz
[    0.000000] clocksource: MIPS: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041786 ns
[    0.000002] sched_clock: 32 bits at 250MHz, resolution 4ns, wraps every 8589934590ns
[    0.015557] Calibrating delay loop... 249.44 BogoMIPS (lpj=1247232)
[    0.097816] pid_max: default: 32768 minimum: 301
[    0.116210] Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.130661] Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    0.157643] RCU Tasks Trace: Setting shift to 0 and lim to 1 rcu_task_cb_adjust=1.
[    0.181080] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.200605] futex hash table entries: 256 (order: -1, 3072 bytes, linear)
[    0.214221] pinctrl core: initialized pinctrl subsystem
[    0.227539] NET: Registered PF_NETLINK/PF_ROUTE protocol family
[    0.246917] rt3883-pci 10140000.pci: mode: PCIe
[    1.901890] rt2880_gpio 10000600.gpio: registering 24 gpios
[    1.912983] rt2880_gpio 10000600.gpio: registering 24 irq handlers
[    1.925201] gpio gpiochip0: Static allocation of GPIO base is deprecated, use dynamic allocation.
[    1.945703] PCI host bridge to bus 0000:00
[    1.953791] pci_bus 0000:00: root bus resource [mem 0x20000000-0x2fffffff]
[    1.967404] pci_bus 0000:00: root bus resource [io  0x10160000-0x1016ffff]
[    1.981075] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    1.997095] pci 0000:00:01.0: [1814:0802] type 01 class 0x060400
[    2.008963] pci 0000:00:01.0: reg 0x14: [mem 0x20100000-0x2010ffff]
[    2.021463] pci 0000:00:01.0: supports D1
[    2.029336] pci 0000:00:01.0: PME# supported from D0 D1 D3hot
[    2.043250] pci 0000:01:00.0: [1814:3091] type 00 class 0x028000
[    2.055154] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x0000ffff]
[    2.067677] pci 0000:01:00.0: PME# supported from D0 D3hot
[    2.078897] pci_bus 0000:01: busn_res: [bus 01-ff] end is updated to 01
[    2.091988] pci_bus 0000:00: busn_res: [bus 00-ff] end is updated to 01
[    2.105135] pci 0000:00:01.0: BAR 8: assigned [mem 0x20000000-0x200fffff]
[    2.118607] pci 0000:00:01.0: BAR 1: assigned [mem 0x20100000-0x2010ffff]
[    2.132121] pci 0000:01:00.0: BAR 0: assigned [mem 0x20000000-0x2000ffff]
[    2.145614] pci 0000:00:01.0: PCI bridge to [bus 01]
[    2.155457] pci 0000:00:01.0:   bridge window [mem 0x20000000-0x200fffff]
[    2.169630] clocksource: Switched to clocksource MIPS
[    2.193924] NET: Registered PF_INET protocol family
[    2.203948] IP idents hash table entries: 2048 (order: 2, 16384 bytes, linear)
[    2.219543] tcp_listen_portaddr_hash hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    2.236332] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear)
[    2.251665] TCP established hash table entries: 1024 (order: 0, 4096 bytes, linear)
[    2.266902] TCP bind hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    2.280943] TCP: Hash tables configured (established 1024 bind 1024)
[    2.293798] UDP hash table entries: 256 (order: 0, 4096 bytes, linear)
[    2.306777] UDP-Lite hash table entries: 256 (order: 0, 4096 bytes, linear)
[    2.321665] NET: Registered PF_UNIX/PF_LOCAL protocol family
[    2.332988] PCI: CLS 0 bytes, default 32
[    2.341060] rt-timer 10000100.timer: maximum frequency is 5065Hz
[    2.361241] workingset: timestamp_bits=30 max_order=14 bucket_order=0
[    2.374608] squashfs: version 4.0 (2009/01/31) Phillip Lougher
[    2.386133] jffs2: version 2.2 (NAND) (SUMMARY) (LZMA) (RTIME) (CMODE_PRIORITY) (c) 2001-2006 Red Hat, Inc.
[    2.411062] Serial: 8250/16550 driver, 2 ports, IRQ sharing disabled
[    2.425964] of_serial: probe of 10000c00.uartlite failed with error -22
[    2.456202] spi spi0.0: force spi mode3
[    2.464477] spi-nor spi0.0: mx25l6405d (8192 Kbytes)
[    2.474428] 5 fixed-partitions partitions found on MTD device spi0.0
[    2.487007] OF: Bad cell count for /palmbus@10000000/spi@b00/flash@0/partitions
[    2.501528] OF: Bad cell count for /palmbus@10000000/spi@b00/flash@0/partitions
[    2.516430] Creating 5 MTD partitions on "spi0.0":
[    2.525951] 0x000000000000-0x000000030000 : "uboot"
[    2.540405] 0x000000030000-0x000000040000 : "uboot-env"
[    2.553856] 0x000000040000-0x000000050000 : "factory"
[    2.567038] OF: Bad cell count for /palmbus@10000000/spi@b00/flash@0/partitions
[    2.581996] 0x000000050000-0x0000007f0000 : "firmware"
[    2.596041] 2 uimage-fw partitions found on MTD device firmware
[    2.607770] Creating 2 MTD partitions on "firmware":
[    2.617585] 0x000000000000-0x0000001c038a : "kernel"
[    2.627441] mtd: partition "kernel" doesn't end on an erase/write block -- force read-only
[    2.646874] 0x0000001c038a-0x0000007a0000 : "rootfs"
[    2.656717] mtd: partition "rootfs" doesn't start on an erase/write block boundary -- force read-only
[    2.677965] mtd: setting mtd5 (rootfs) as root device
[    2.688079] 1 squashfs-split partitions found on MTD device rootfs
[    2.700272] 0x0000006f0000-0x0000007a0000 : "rootfs_data"
[    2.714021] 0x0000007f0000-0x000000800000 : "user-cfg"
[    2.734959] rtl8367b rtl8367b: cannot find mdio node phandle
[    2.746192] rtl8367b rtl8367b: using GPIO pins 1 (SDA) and 2 (SCK)
[    2.759610] rtl8367b rtl8367b: RTL8367R-VB chip found
[    2.769549] rtl8367b rtl8367b: CPU port: 5
[    4.650658] mtk_soc_eth 10100000.ethernet: generated random MAC address 62:fd:ca:e9:36:f7
[    4.666900] mtk_soc_eth 10100000.ethernet: mdio-bus disabled
[    4.678119] mtk_soc_eth 10100000.ethernet: using fixed link parameters
[    4.691077] mtk_soc_eth 10100000.ethernet eth0 (uninitialized): link up (1000Mbps/Full duplex)
[    4.709288] mtk_soc_eth 10100000.ethernet eth0: mediatek frame engine at 0xb0100000, irq 5
[    4.726579] rt2880_wdt 10000120.watchdog: Initialized
[    4.739227] NET: Registered PF_INET6 protocol family
[    4.758015] Segment Routing with IPv6
[    4.765379] In-situ OAM (IOAM) with IPv6
[    4.773375] NET: Registered PF_PACKET protocol family
[    4.783362] 8021q: 802.1Q VLAN Support v1.8
[    4.842779] clk: Disabling unused clocks
[    4.850626] Warning: unable to open an initial console.
[    4.875802] VFS: Mounted root (squashfs filesystem) readonly on device 31:5.
[    4.898308] Freeing unused kernel image (initmem) memory: 1180K
[    4.910007] This architecture does not have kernel memory protection.
[    4.922792] Run /sbin/init as init process
[    9.100441] init: Console is alive
[    9.107750] init: - watchdog -
[   12.949596] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[   13.621387] usbcore: registered new interface driver usbfs
[   13.632385] usbcore: registered new interface driver hub
[   13.643067] usbcore: registered new device driver usb
[   13.720514] gpio_button_hotplug: loading out-of-tree module taints kernel.
[   13.763598] loop: module loaded
[   13.794643] SCSI subsystem initialized
[   13.829572] phy phy-usbphy.0: remote usb device wakeup disabled
[   13.841263] phy phy-usbphy.0: UTMI 16bit 30MHz
[   13.850089] ehci-platform 101c0000.ehci: EHCI Host Controller
[   13.861554] ehci-platform 101c0000.ehci: new USB bus registered, assigned bus number 1
[   13.877471] ehci-platform 101c0000.ehci: irq 26, io mem 0x101c0000
[   13.919661] ehci-platform 101c0000.ehci: USB 2.0 started, EHCI 1.00
[   13.933667] hub 1-0:1.0: USB hub found
[   13.942573] hub 1-0:1.0: 2 ports detected
[   13.967233] ohci-platform 101c1000.ohci: Generic Platform OHCI controller
[   13.980768] ohci-platform 101c1000.ohci: new USB bus registered, assigned bus number 2
[   13.996681] ohci-platform 101c1000.ohci: irq 26, io mem 0x101c1000
[   14.075928] hub 2-0:1.0: USB hub found
[   14.084280] hub 2-0:1.0: 2 ports detected
[   14.119495] usbcore: registered new interface driver usb-storage
[   14.136122] usbcore: registered new interface driver uas
[   14.148645] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[   14.173784] init: - preinit -
[   19.719678] random: crng init done
[   23.554034] mtk_soc_eth 10100000.ethernet eth0: link up (1000Mbps/Full duplex)
[   23.569230] 8021q: adding VLAN 0 to HW filter on device eth0
[   24.310205] platform keys: deferred probe pending
[   27.644173] jffs2: notice: (400) jffs2_build_xattr_subsystem: complete building xattr subsystem, 17 of xdatum (0 unchecked, 6 orphan) and 20 of xref (4 dead, 2 orphan) found.
[   27.677756] mount_root: switching to jffs2 overlay
[   27.696533] overlayfs: upper fs does not support tmpfile.
[   27.724123] urandom-seed: Seeding with /etc/urandom.seed
[   28.195337] mtk_soc_eth 10100000.ethernet eth0: link down
[   28.246099] procd: - early -
[   28.253111] procd: - watchdog -
[   30.010021] procd: - watchdog -
[   30.016413] procd: failed to set stdio: No such device
[   30.051163] procd: - ubus -
[   31.912405] procd: - init -
[   36.407460] kmodloader: loading kernel modules from /etc/modules.d/*
[   37.833379] usbcore: registered new interface driver cdc_wdm
[   37.880887] Loading modules backported from Linux version v6.6.15-0-g51f354b815c4
[   37.895726] Backport generated by backports.git 193becf2
[   38.044927] usbcore: registered new interface driver usbserial_generic
[   38.058001] usbserial: USB Serial support registered for generic
[   38.142364] usbcore: registered new interface driver cdc_ether
[   38.183492] usbcore: registered new interface driver cdc_ncm
[   38.360641] usbcore: registered new interface driver huawei_cdc_ncm
[   38.541118] urngd: v1.0.2 started.
[   38.855678] usbcore: registered new interface driver option
[   38.866851] usbserial: USB Serial support registered for GSM modem (1-port)
[   38.992628] rt2800pci 0000:01:00.0: enabling device (0000 -> 0002)
[   39.005110] ieee80211 phy0: rt2x00_set_rt: Info - RT chipset 3071, rev 021c detected
[   39.023937] ieee80211 phy0: rt2x00_set_rf: Info - RF chipset 0001 detected
[   39.210302] ieee80211 phy1: rt2x00_set_rt: Info - RT chipset 3883, rev 0400 detected
[   39.226307] ieee80211 phy1: rt2x00_set_rf: Info - RF chipset 3853 detected
[   40.640217] kmodloader: done loading kernel modules from /etc/modules.d/*
[   90.906652] mtk_soc_eth 10100000.ethernet eth0: link up (1000Mbps/Full duplex)
[   90.922869] 8021q: adding VLAN 0 to HW filter on device eth0
[   91.276732] mtk_soc_eth 10100000.ethernet eth0: entered promiscuous mode
[   91.312169] br-lan: port 1(eth0.1) entered blocking state
[   91.322864] br-lan: port 1(eth0.1) entered disabled state
[   91.333615] eth0.1: entered allmulticast mode
[   91.342227] mtk_soc_eth 10100000.ethernet eth0: entered allmulticast mode
[   91.356135] eth0.1: entered promiscuous mode
[   91.438670] br-lan: port 1(eth0.1) entered blocking state
[   91.449368] br-lan: port 1(eth0.1) entered forwarding state
[  128.059906] ieee80211 phy0: rt2x00lib_request_firmware: Info - Loading firmware file 'rt2860.bin'
[  130.121866] ieee80211 phy0: rt2x00lib_request_firmware: Info - Firmware detected - version: 0.40
[  130.511752] br-lan: port 2(phy0-ap0) entered blocking state
[  130.522821] br-lan: port 2(phy0-ap0) entered disabled state
[  130.533904] rt2800pci 0000:01:00.0 phy0-ap0: entered allmulticast mode
[  130.547303] rt2800pci 0000:01:00.0 phy0-ap0: entered promiscuous mode
[  131.375257] br-lan: port 2(phy0-ap0) entered blocking state
[  131.386307] br-lan: port 2(phy0-ap0) entered forwarding state
[  132.899608] br-lan: port 3(phy1-ap0) entered blocking state
[  132.910710] br-lan: port 3(phy1-ap0) entered disabled state
[  132.921920] rt2800_wmac 10180000.wmac phy1-ap0: entered allmulticast mode
[  132.935801] rt2800_wmac 10180000.wmac phy1-ap0: entered promiscuous mode
[  134.133961] br-lan: port 3(phy1-ap0) entered blocking state
[  134.145030] br-lan: port 3(phy1-ap0) entered forwarding state

I have extracted the squashfs of my sysupgrade image and it does contain the files as intended. And I have used the -n parameter of sysupgrade so I have no idea why they are not applied. The kernel source related to the jffs2 warning shortly before the reboot does not read exactly great. Could that be the reason for the missing custom files?

The warning immediately at boot time from the clock configuration is probably benign as the correct settings are applied eventually.

What could be the reason for the missing shell? The message procd: failed to set stdio: No such device seems very suspicious and stems from here. The default dts for the rt3888 sets bootargs = "console=ttyS0,57600"; which is reflected in the boot log. After boot the ttyS0 does indeed exist (among others):

# ls -l /dev/tty*
crw-rw-rw-    1 root     root        5,   0 Jun 27 00:18 /dev/tty
crw-rw----    1 root     dialout     4,  64 Jan  1  1970 /dev/ttyS0
crw-rw----    1 root     dialout     4,  65 Jan  1  1970 /dev/ttyS1

tty is the current virtual shell AFAICT, not sure about the other two. I cant access them ATM (at least not with cat and echo and there no terminal program not stty available). Maybe they are related to the support for (USB) WWAN I compiled in? Could there be a conflict between those and the hardware UART?

Any hints on how to debug/fix these problems welcome

RaLink wireless hardware and the rt2x00 drivers have never been 'great', support only improved (massively) after Mediatek took over (and another time for chipsets starting with mt7615). While that doesn't directly help you, it's something to keep in mind (it 'should' work, but not as reliably as one would hope).

I know they have a bad rep but I had basically no issues whatsoever with this hardware in the past... apart from the fact that my second device just stopped receiving ethernet packets on the wired ports two weeks ago :slight_smile: I wouldn't have touched the (Padawan) firmware without this hardware issue but since I had to set up the backup device I would definitely prefer to use an up-to-date distro. I wasn't even aware that @kapieyow had ported OpenWRT to it 5 years ago before that failure :slight_smile:

sysupgrade -n means no files from original install will be preserved. Even less anything will be added to squash fs on the fly.

overlay unmount message is scary but harmless especially since you wipe it right away.