Banana BPI-R3 configuration not being saved

Hi. I'm starting my journey with OpenWrt and for this I've choose the Banana BPI-R3. I've started with default snapshot build but I wanted to make it more fitted to my needs so I've started with custom image build. I've managed to prepare building environment on Ubuntu 23.04 VM and after some trials and errors I've managed to get my first build. I'm using the eMMC, so to flash it I had to first prepare SD card with image, next write it to NAND flash, and next copy it from NAND to eMMC, and this seemed to work just fine. But for next builds I've used the squashfs-sysupgrade.itb flashing through LuCI. And now I'm observing weird behavior. When I alter the configuration and next flash system update through LuCI, this altered configuration is stored and remembered - I can reboot/power off and it doesn't revert anything. But if I next change something (doesn't matter if it was done through LuCI or UART/ssh), then configuration is stored until first reboot - after power off/reboot those changes are reverted. I'm not quite sure why. Could someone help me understand why this is happening?

root@OpenWrt:/# uname -a
Linux OpenWrt 5.15.109 #0 SMP Sun Apr 30 16:27:00 2023 aarch64 GNU/Linux

root@OpenWrt:/# cat /etc/os-release
NAME="OpenWrt"
VERSION="SNAPSHOT"
ID="openwrt"
ID_LIKE="lede openwrt"
PRETTY_NAME="OpenWrt SNAPSHOT"
VERSION_ID="snapshot"
HOME_URL="https://openwrt.org/"
BUG_URL="https://bugs.openwrt.org/"
SUPPORT_URL="https://forum.openwrt.org/"
BUILD_ID="r22701-3221128639"
OPENWRT_BOARD="mediatek/filogic"
OPENWRT_ARCH="aarch64_cortex-a53"
OPENWRT_TAINTS="busybox"
OPENWRT_DEVICE_MANUFACTURER="OpenWrt"
OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/"
OPENWRT_DEVICE_PRODUCT="Generic"
OPENWRT_DEVICE_REVISION="v0"
OPENWRT_RELEASE="OpenWrt SNAPSHOT r22701-3221128639"

root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                22.3M     22.3M         0 100% /rom
tmpfs                   996.4M    980.0K    995.5M   0% /tmp
tmpfs                   996.4M    192.0K    996.2M   0% /tmp/root
overlayfs:/tmp/root     996.4M    192.0K    996.2M   0% /
tmpfs                   512.0K         0    512.0K   0% /dev

EDIT : Ok, so it seems that rubber duck debugging worked again like a charm :smiley: Just after publishing my thread I noticed that my df output looks weird, so I checked how it was before, and it was different :

root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                 5.0M      5.0M         0 100% /rom
tmpfs                   996.4M      1.0M    995.4M   0% /tmp
/dev/mmcblk0p66        1011.8M    160.9M    850.9M  16% /overlay
overlayfs:/overlay     1011.8M    160.9M    850.9M  16% /
tmpfs                   512.0K         0    512.0K   0% /dev

So it seems that now everything it stored in RAM instead of flash. I'll try to flash again through SD->NAND->eMMC again and check if it's still the same.

EDIT2: Ok, it seems that it's the same even for SD card. I've checked dmesg and found something interesting :

root@OpenWrt:/# dmesg | grep mmc
[    0.000000] Kernel command line: root=/dev/mmcblk0p65
[    1.258654] mmc0: host does not support reading read-only switch, assuming write-enable
[    1.275327] mmc0: new high speed SDHC card at address aaaa
[    1.284435] mmcblk0: mmc0:aaaa SB32G 29.7 GiB
[    1.357652] FIT:          flat_dt sub-image 0x0053d000..0x00544fff "fdt-mt7986a-bananapi-bpi-r3-emmc-nor" (ARM64 OpenWrt bananapi_bpi-r3 device tree overlay mt7986a-bananapi-bpi-r3-emmc-nor)
[    1.374661] FIT:          flat_dt sub-image 0x00545000..0x0054cfff "fdt-mt7986a-bananapi-bpi-r3-emmc-snand" (ARM64 OpenWrt bananapi_bpi-r3 device tree overlay mt7986a-bananapi-bpi-r3-emmc-snand)
[    1.426377]  mmcblk0: p1 p2 p3 p4 p5 p6 p7 p65(rootfs-1) p66(rootfs_data)
[    6.007946] FIT:          flat_dt sub-image 0x0053d000..0x00544fff "fdt-mt7986a-bananapi-bpi-r3-emmc-nor" (ARM64 OpenWrt bananapi_bpi-r3 device tree overlay mt7986a-bananapi-bpi-r3-emmc-nor)
[    6.024954] FIT:          flat_dt sub-image 0x00545000..0x0054cfff "fdt-mt7986a-bananapi-bpi-r3-emmc-snand" (ARM64 OpenWrt bananapi_bpi-r3 device tree overlay mt7986a-bananapi-bpi-r3-emmc-snand)
**[    9.358434] mount_root: overlay filesystem in /dev/mmcblk0p66 has not been formatted yet**
[   16.617892] MTD: Attempt to mount non-MTD device "/dev/mmcblk0p66"

Output from fdisk:

root@OpenWrt:/# fdisk -l
[  106.457900] mtdblock: MTD device 'ubi' is NAND, please consider using UBI blo                                                                                                                                                                                                                                             ck devices instead.
[  106.466733] mtdblock: MTD device 'fip' is NAND, please consider using UBI blo                                                                                                                                                                                                                                             ck devices instead.
[  106.475521] mtdblock: MTD device 'reserved' is NAND, please consider using UB                                                                                                                                                                                                                                             I block devices instead.
[  106.484733] mtdblock: MTD device 'bl2' is NAND, please consider using UBI blo                                                                                                                                                                                                                                             ck devices instead.
Found valid GPT with protective MBR; using GPT

Disk /dev/mmcblk0: 62333952 sectors, 1764M
Logical sector size: 512
Disk identifier (GUID): 5452574f-2211-4433-5566-778899aabb00
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 4325375

Number  Start (sector)    End (sector)  Size Name
     1              34            8191 4079K bl2
     2            8192            9215  512K ubootenv
     3            9216           13311 2048K factory
     4           13312           21503 4096K fip
     5           24576           90111 32.0M recovery
     6           90112          131071 20.0M install
     7          131072         4325375 2048M production
Disk /dev/ubiblock0_2: 27 MB, 28950528 bytes, 56544 sectors
252 cylinders, 1 heads, 224 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/ubiblock0_2 doesn't contain a valid partition table
Disk /dev/ubiblock0_2p1: 22 MB, 23265280 bytes, 45440 sectors
202 cylinders, 1 heads, 224 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Disk /dev/ubiblock0_2p1 doesn't contain a valid partition table
Disk /dev/nvme0n1: 13 GB, 14403239936 bytes, 28131328 sectors
219776 cylinders, 4 heads, 32 sectors/track
Units: sectors of 1 * 512 = 512 bytes

Device       Boot StartCHS    EndCHS        StartLBA     EndLBA    Sectors  Size                                                                                                                                                                                                                                              Id Type
/dev/nvme0n1p1    256,0,1     1023,3,32        32768    8011775    7979008 3896M                                                                                                                                                                                                                                              83 Linux

Ok, I think I've found the issue and it seems to be a BUG in OpenWrt configuration for mediatek/filogic. The problem is that mkf2fs is missing, so the initial mmcblk0p66 can't be formatted.

[    8.788610] mount_root: overlay filesystem in /dev/mmcblk0p66 has not been formatted yet
sh: mkfs.f2fs: not found
[    8.800660] mount_root: jffs2 not ready yet, using temporary tmpfs overlay

It (sh: mkfs.f2fs: not found) wasn't visible in dmesg, but only on UART during boot.

I've checked and mkf2fs is not build when only one target device is selected (in my case I've selected only BPI-R3 to make builds faster), because packages has dependency on TARGET_MULTI_PROFILE:

 Symbol: MODULE_DEFAULT_f2fsck [=y]                                                                                               x
  x Type  : tristate                                                                                                                 x
  x Defined at tmp/.config-target.in:247351                                                                                          x
  x   Depends on: TARGET_PER_DEVICE_ROOTFS [=y] && m && MODULES [=y]   < Help >                                                      x
  x Selects: PACKAGE_f2fsck [=m]                                                                                                     x
  x Selected by [y]:                                                                                                                 x
  x   - TARGET_DEVICE_mediatek_filogic_DEVICE_bananapi_bpi-r3 [=y] && TARGET_MULTI_PROFILE [=y] && TARGET_mediatek_filogic [=y] && T x
  x Selected by [n]:                                                                                                                 x
  x   - TARGET_DEVICE_apm821xx_sata_DEVICE_wd_mybooklive [=n] && TARGET_MULTI_PROFILE [=y] && TARGET_apm821xx_sata [=n] && TARGET_PE x
  x   - TARGET_DEVICE_apm821xx_sata_DEVICE_wd_mybooklive [=n] && TARGET_MULTI_PROFILE [=y] && TARGET_apm821xx_sata [=n] && TARGET_PE x
  x   - TARGET_DEVICE_mediatek_mt7622_DEVICE_bananapi_bpi-r64 [=n] && TARGET_MULTI_PROFILE [=y] && TARGET_mediatek_mt7622 [=n] && TA x
  x                                                                                                                                  x
  x                                                                                                                                  x
  x Symbol: PACKAGE_f2fsck [=m]                                                                                                      x
  x Type  : tristate                                                                                                                 x
  x Defined at tmp/.config-package.in:123849                                                                                         x
  x   Prompt: f2fsck                                                                                                                 x
  x   Depends on: m && MODULES [=y] || PACKAGE_f2fsck-selinux [=n]!=y                                                                x
  x   Location:                                                                                                                      x
  x     -> Utilities                                                                                                                 x
  x (1)   -> Filesystem                                                                                                              x
  x Selects: PACKAGE_libc [=y] && PACKAGE_libf2fs [=m]                                                                               x
  x Selected by [m]:                                                                                                                 x
  x   - MODULE_DEFAULT_f2fsck [=y] && TARGET_PER_DEVICE_ROOTFS [=y] && m && MODULES [=y]                                             x
  x Selected by [n]:                                                                                                                 x
  x   - PACKAGE_f2fs-tools [=n] && (m && MODULES [=y] || PACKAGE_f2fs-tools-selinux [=n]!=y) 

EDIT : Now I can confirm, that changing Target Profile to Multiple Devices resolves the issue:

root@OpenWrt:/# df -h
Filesystem                Size      Used Available Use% Mounted on
/dev/root                24.0M     24.0M         0 100% /rom
tmpfs                   996.4M    340.0K    996.1M   0% /tmp
/dev/mmcblk0p66         480.8M    111.0M    369.8M  23% /overlay
overlayfs:/overlay      480.8M    111.0M    369.8M  23% /
tmpfs                   512.0K         0    512.0K   0% /dev

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.