FW ENV broken on ESPRESSOBIN-ULTRA (21.02-RC3)

For Developers
1

May I have to open an issue on this ?

root@ultra:~# fw_printenv 
Warning: Bad CRC, using default environment
bootcmd=bootp; setenv bootargs root=/dev/nfs nfsroot=${serverip}:${rootpath} ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}:${hostname}::off; bootm
bootdelay=5
baudrate=115200

root@ultra:~# cat /proc/mtd 
dev:    size   erasesize  name
mtd0: 003e0000 00010000 "firmware"
mtd1: 00010000 00010000 "hw-info"
mtd2: 00010000 00010000 "u-boot-env"

root@ultra:~# cat /etc/fw_env.config 
/dev/mtd2 0x0 0x10000 0x10000 1

root@ultra:~# cat /etc/board.json 
{
	"model": {
		"id": "globalscale,espressobin-ultra",
		"name": "Globalscale Marvell ESPRESSOBin Ultra Board"
	},
	"network": {
		"lan": {
			"ports": [
				"lan0",
				"lan1",
				"lan2",
				"lan3"
			],
			"protocol": "static"
		},
		"wan": {
			"device": "wan",
			"protocol": "dhcp"
		}
	}
}

I just reboot to u-boot and save env... boot again in OpenWrt 21.02-RC3 and still get the same CRC checksum error message on fw_printenv !
The problem in that case is that fw_saveenv will broke the u-boot env...

Link : already reported in the master 21.02-RC3 topic :

2

dmesg

[    0.966740] spi-nor spi0.0: mx25u3235f (4096 Kbytes)
[    0.971897] 3 fixed-partitions partitions found on MTD device spi0.0
[    0.978390] Creating 3 MTD partitions on "spi0.0":
[    0.983295] 0x000000000000-0x0000003e0000 : "firmware"
[    0.989205] 0x0000003e0000-0x0000003f0000 : "hw-info"
[    0.994913] 0x0000003f0000-0x000000400000 : "u-boot-env"
3

no... go back to the start...

  • what was factory? ( the previous OS )
  • did you upgrade and keep settings?
  • did you manually alter the rootfs in any way or use uboot during the upgrade?
  • is that normal to do so for that device?

"here are my settings" and "this is what I ran in uboot" is minimal at best... and speaks very little: re causation...

if you have difficulty with the above...

restore a functional env and reproduce steps, documenting as you go...

4 
TIM-1.0                                                                                                                                                               
WTMI-devel-18.12.1-67f01b7                                                                                                                                            
WTMI: system early-init                                                                                                                                               
SVC REV: 5, CPU VDD voltage: 1.213V                                                                                                                                   
NOTICE:  Booting Trusted Firmware                                                                                                                                     
NOTICE:  BL1: v1.5(release):711ecd32 (Marvell-armada-18.09.4)                                                                                                         
NOTICE:  BL1: Built : 15:20:15, Sep 18 2019                                                                                                                           
NOTICE:  BL1: Booting BL2                                                                                                                                             
NOTICE:  BL2: v1.5(release):711ecd32 (Marvell-armada-18.09.4)                                                                                                         
NOTICE:  BL2: Built : 15:20:18, Sep 18 2019                                                                                                                           
NOTICE:  BL1: Booting BL31                                                                                                                                            
NOTICE:  BL31: v1.5(release):711ecd32 (Marvell-armada-18.09.4)                                                                                                        
NOTICE:  BL31: Built : 15                                                                                                                                             
                                                                                                                                                                      
U-Boot 2017.03-armada-18.09.1-g51aa6c4772 (Sep 18 2019 - 15:19:13 +0800)                                                                                              
                                                                                                                                                                      
Model: gti cellular cpe board                                                                                                                                         
       CPU     1200 [MHz]                                                                                                                                             
       L2      1200 [MHz]                                                                                                                                             
       NB AXI  300 [MHz]                                                                                                                                              
       SB AXI  250 [MHz]                                                                                                                                              
       DDR     750 [MHz]                                                                                                                                              
DRAM:  1 GiB                                                                                                                                                          
U-Boot DT blob at : 000000003f716f38                                                                                                                                  
SF: Detected mx25u3235f with page size 256 Bytes, erase size 64 KiB, total 4 MiB                                                                                      
Comphy chip #0:                                                                                                                                                       
Comphy-0: USB3_HOST0                                                                                                                                                  
Comphy-1: PEX0          2.5 Gbps                                                                                                                                      
Comphy-2: SATA0                                                                                                                                                       
Target spinup took 0 ms.                                                                                                                                              
AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode                                                                                                             
flags: ncq led only pmp fbss pio slum part sxs                                                                                                                        
PCIE-0: Link up                                                                                                                                                       
MMC:   sdhci@d8000: 0                                                                                                                                                 
Net:   eth0: neta@30000 [PRIME]                                                                                                                                       
Hit any key to stop autoboot:  0                                                                                                                                      
Marvell>>
5

sorry, no problem, nor corruption by saving and booting from uboot...
The problem IS that OpenWrt do not take care correctly of the uboot env, as it will in standard EspressoBin v7 with OpenWrt 19.07.x !
The problem IS on EspresoBin Ultra on OpenWrt 21.02-RC3 !
Not tested EspressoBin v7 on OpenWrt 21.02-RC3 for now... to see if they have the problems also...

The problem is :
ENV settings not managed correctly, and it may results in broking the system if writing env from OpenWrt !

This is a factory (aka default) uboot...

Find some references on the GTI GITHUB :

I still searching the reason...

6

Default EBINv7 configs get same problem with reading ENV; CRC Checksum error...
Tested in the EBINULTRA on OpenWrt 21.02-RC3

root@ultra:~# cat /etc/fw_env.config 
##/dev/mtd2 0x0 0x10000 0x10000 1
/dev/mtd0 0x3f0000 0x10000 0x10000 1

May be the problem is with :

/* Environment in MMC */
#define CONFIG_SYS_MMC_ENV_DEV		1 /* 0 means sdio, 1 means mmc. */
7

Are these numbers correct? I.e., is the size of the checksummed environment 0x10000, or is it using a smaller part of the partition/block?

Easiest to check using hexdump IMHO. Try to make some sense out of hexdump -C /dev/mtd2. The environment ends where the 00's end and the ff's start.

8

No, and maybe spi is not storing the ENV at all...

root@ultra:~# hexdump -C /dev/mtd2 > mtd2.hex

and looking at mtd2.hex but not found any strings at all which may being in the envs settings !

root@ultra:~# strings /dev/mtd2 | grep bootcmd

give also no results at all !

9

it been zapped you need to restore...

10

It have not happen... but may if ! so I report the issue of env not being correct on ULTRA EspressoBin Board type... which is added in 21.02

11

Sounds likely. If u-boot doesn't store the enviroment in /dev/mtd2 then it should not come as a suprise that there is nothing in there...

Is this the default configuration for these boards? If so, then I guess the uboot-envtools definition should be updated. Or ideally, it should be fixed to auto-detect this on first boot.

1 Like
12

it is... yes... I never flashed uboot on this board !

13

Looks like the env settins of OpenWrt were taken from : https://patchwork.ozlabs.org/project/uboot/list/?submitter=78214

But no uboot binaries available for EbinUltra on OpenWrt snapshot downloads...

14

flashed for testing with the latest uboot binary from GTI :

U-Boot 2018.03-devel-18.12.3-g057aa3fce1 (Jun 16 2020 - 15:35:51 +0800)

still same issue !

15

figure out where the env is stored on the mmc and update /etc/fw_env.config accordingly. When you have it working, submit a patch to fix the defaults in package/boot/uboot-envtools/files/mvebu

1 Like
16

This is a bug with the SPI-NOR config in the DTS.
I have patches that I will start sending soon for the Ultra as its pretty much fully broken at this point,
waiting on some more info from Globalscale

1 Like
17

If you need a beta tester, I always pleased to get some free time to help... :wink:

18

I just attempted to flash the latest stable openwrt onto a new espressobin-ultra and it's not booting. The internal flash had linux on it. Does the uboot env need to be updated to work with the openwrt image?

TIM-1.0   
WTMI-devel-18.12.0-1421e17
WTMI: system early-init
SVC REV: 5, CPU VDD voltage: 1.213V
NOTICE:  Booting Trusted Firmware
NOTICE:  BL1: v1.5(release):26d6b50f (Marvell-devel-18.12.2)
NOTICE:  BL1: Built : 15:51:12, Mar  4 2021
NOTICE:  BL1: Booting BL2
NOTICE:  BL2: v1.5(release):26d6b50f (Marvell-devel-18.12.2)
NOTICE:  BL2: Built : 15:51:17, Mar  4 2021
NOTICE:  BL1: Booting BL31
NOTICE:  BL31: v1.5(release):26d6b50f (Marvell-devel-18.12.2)
NOTICE:  BL31: Built : 15

U-Boot 2018.03-devel-18.12.3-g551329ec74 (Mar 04 2021 - 15:48:05 +0800)

Model: gti cellular cpe board
       CPU     1200 [MHz]
       L2      800 [MHz]
       TClock  200 [MHz]
       DDR     750 [MHz]
DRAM:  1 GiB
SF: Detected mx25u3235f with page size 256 Bytes, erase size 64 KiB, total 4 MiB
Comphy chip #0:
Comphy-0: USB3_HOST0
Comphy-1: PEX0          2.5 Gbps
Comphy-2: SATA0
SATA link 0 timeout.
AHCI 0001.0300 32 slots 1 ports 6 Gbps 0x1 impl SATA mode
flags: ncq led only pmp fbss pio slum part sxs
PCIE-0: Link up
MMC:   sdhci@d8000: 0
Loading Environment from SPI Flash... OK
Model: gti cellular cpe board
Net:   eth0: neta@30000 [PRIME]
Hit any key to stop autoboot:  0
switch to partitions #0, OK
mmc0(part 0) is current device
** File not found boot/Image **
** File not found boot/armada-3720-ccpe.dtb **
## Flattened Device Tree blob at 06f00000
   Booting using the fdt blob at 0x6f00000
   reserving fdt memory region: addr=6f00000 size=4000
   Using Device Tree in place at 0000000006f00000, end 0000000006f06fff
   
Starting kernel ...

"Synchronous Abort" handler, esr 0x02000000
elr: ffffffffc7ab4000 lr : 0000000000001fe8 (reloc)
elr: 00000000079f0000 lr : 000000003ff3dfe8
x0 : 0000000006f00000 x1 : 0000000000000000
x2 : 0000000000000000 x3 : 0000000000000000
x4 : 0000000007000000 x5 : 0000000000000001
x6 : 0000000000000008 x7 : 0000000000000000
x8 : 000000003f900000 x9 : 0000000000000002
x10: 000000000a200023 x11: 0000000000000002
x12: 0000000000000002 x13: 00000000000000c0
x14: 000000003f628a1c x15: 000000003ff3d42c
x16: 000000003ffbc4d0 x17: 000000003ffbc4d0
x18: 000000003f62bdb0 x19: 000000003ffd3b20
x20: 0000000000000000 x21: 0000000000000000
x22: 0000000000000003 x23: 000000003f6b28b8
x24: 000000003ffc2960 x25: 0000000000000000
x26: 000000003ff3e010 x27: 0000000000000000
x28: 0000000000000400 x29: 000000003f628b20

Resetting CPU ...

resetting ...
19

From my notes, a quick answer:
This is how I force a flash and automatic boot to eMMC.

FROM UBOOT

setenv bootowrt 'mmc dev 0; ext4load mmc 0:1 $kernel_addr_r $image_name; ext4load mmc 0:1 $fdt_addr_r $fdt_name; setenv bootargs $console root=/dev/mmcblk0p2 rw rootwait net.ifnames=0 biosdevname=0  $extra_params usb-storage.quirks=$usbstoragequirks; booti $kernel_addr_r - $fdt_addr_r'
setenv bootcmd 'run bootowrt'
setenv fdt_name 'armada-3720-espressobin-ultra.dtb'
setenv image_name 'Image'

saveenv
reset
20

Any word on this? I am facing the same issue.

Also have you noticed that there is a .dts file called armada-3720-espressobin-ultra.dts in the linux mainline kernel now. I tried it and it also wasn't 100% working (wifi broken)