Both WLAN radios down: UBIFS error ... bad key order

Installing and Using OpenWrt
1

After modifying /etc/config/wireless and /etc/config/network to set up gretap tunnel, both radios fail to come up. Initially I thought it's due to the fake MAC addrs on the radios, but wifi remains kaputt after reverting to default MACs.

logread

Sat Dec  4 05:22:50 2021 user.notice mac80211: Failed command: iw phy phy1 set antenna 0xffffffff 0xffffffff
Sat Dec  4 05:22:50 2021 daemon.notice netifd: radio1 (5887): command failed: Not supported (-122)
Sat Dec  4 05:22:50 2021 user.notice mac80211: Failed command: iw phy phy1 set distance 0
Sat Dec  4 05:22:50 2021 daemon.err hostapd: Configuration file: /var/run/hostapd-phy1.conf
Sat Dec  4 05:22:50 2021 kern.info kernel: [ 4719.014938] IPv6: ADDRCONF(NETDEV_UP): wlan1: link is not ready
Sat Dec  4 05:22:50 2021 kern.info kernel: [ 4719.027575] br-lan: port 2(wlan1) entered blocking state
Sat Dec  4 05:22:50 2021 kern.info kernel: [ 4719.032957] br-lan: port 2(wlan1) entered disabled state
Sat Dec  4 05:22:50 2021 daemon.notice hostapd: wlan1: interface state UNINITIALIZED->COUNTRY_UPDATE
Sat Dec  4 05:22:50 2021 kern.info kernel: [ 4719.038862] device wlan1 entered promiscuous mode
Sat Dec  4 05:22:50 2021 daemon.notice hostapd: wlan1: interface state COUNTRY_UPDATE->HT_SCAN
Sat Dec  4 05:22:51 2021 daemon.err hostapd: Using interface wlan1 with hwaddr 00:xx:yy:07:ee:74 and ssid " 5ghz"
Sat Dec  4 05:22:51 2021 user.notice mac80211: Failed command: iw phy phy0 set antenna 0xffffffff 0xffffffff
Sat Dec  4 05:22:51 2021 kern.info kernel: [ 4720.099043] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
Sat Dec  4 05:22:51 2021 kern.info kernel: [ 4720.105849] br-lan: port 2(wlan1) entered blocking state
Sat Dec  4 05:22:51 2021 kern.info kernel: [ 4720.111247] br-lan: port 2(wlan1) entered listening state
Sat Dec  4 05:22:51 2021 kern.info kernel: [ 4720.126364] IPv6: ADDRCONF(NETDEV_UP): wtun: link is not ready
Sat Dec  4 05:22:51 2021 daemon.err hostapd: Using interface wtun with hwaddr 00:xx:yy:07:ee:75 and ssid "wifitrunk"
Sat Dec  4 05:22:51 2021 kern.info kernel: [ 4720.214766] IPv6: ADDRCONF(NETDEV_CHANGE): wtun: link becomes ready
Sat Dec  4 05:22:51 2021 daemon.notice hostapd: wlan1: interface state HT_SCAN->ENABLED
Sat Dec  4 05:22:51 2021 daemon.notice hostapd: wlan1: AP-ENABLED
Sat Dec  4 05:22:51 2021 kern.err kernel: [ 4720.296804] UBIFS error (ubi0:1 pid 6117): 0x801d79d4: bad key order (keys 2 and 3)
Sat Dec  4 05:22:51 2021 kern.err kernel: [ 4720.304582] UBIFS error (ubi0:1 pid 6117): 0x801d7a30: bad indexing node at LEB 101:30472, error 6
Sat Dec  4 05:22:51 2021 kern.err kernel: [ 4720.313631] 	magic          0x6101831
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.317411] 	crc            0x99f275e1
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.321354] 	node_type      9 (indexing node)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.325792] 	group_type     0 (no node group)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.330248] 	sqnum          23994
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.333583] 	len            188
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.336782] 	child_cnt      8
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.339837] 	level          0
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.342892] 	Branches:
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.345346] 	0: LEB 98:99440 len 2726 key (2278, data, 57)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.350913] 	1: LEB 98:102168 len 2714 key (2278, data, 58)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.356591] 	2: LEB 98:104888 len 2640 key (2278, data, 59)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.362279] 	3: LEB 98:107528 len 2546 key (2278, inode)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.367671] 	4: LEB 98:110080 len 2420 key (2278, data, 61)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.373377] 	5: LEB 98:112504 len 2677 key (2278, data, 62)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.379007] 	6: LEB 98:115184 len 1971 key (2278, data, 63)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.384661] UBIFS error (ubi0:1 pid 6117): 0x801ad438: cannot read page 57 of inode 2278, error -22
Sat Dec  4 05:22:52 2021 daemon.notice netifd: radio1 (5887): Bus error (core dumped)
Sat Dec  4 05:22:52 2021 daemon.notice netifd: radio1 (5887): Interface 0 setup failed: IFUP_ERROR
Sat Dec  4 05:22:52 2021 daemon.err hostapd: Configuration file: /var/run/hostapd-phy0.conf
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.423950] UBIFS error (ubi0:1 pid 6141): 0x801d79d4: bad key order (keys 2 and 3)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.432043] UBIFS error (ubi0:1 pid 6141): 0x801d7a30: bad indexing node at LEB 101:30472, error 6
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.441179] 	magic          0x6101831
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.444937] 	crc            0x99f275e1
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.448790] 	node_type      9 (indexing node)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.453195] 	group_type     0 (no node group)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.457621] 	sqnum          23994
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.461051] 	len            188
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.464239] 	child_cnt      8
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.467245] 	level          0
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.470359] 	Branches:
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.472836] 	0: LEB 98:99440 len 2726 key (2278, data, 57)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.478467] 	1: LEB 98:102168 len 2714 key (2278, data, 58)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.484121] 	2: LEB 98:104888 len 2640 key (2278, data, 59)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.489768] 	3: LEB 98:107528 len 2546 key (2278, inode)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.495084] 	4: LEB 98:110080 len 2420 key (2278, data, 61)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.500702] 	5: LEB 98:112504 len 2677 key (2278, data, 62)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.506340] 	6: LEB 98:115184 len 1971 key (2278, data, 63)
Sat Dec  4 05:22:52 2021 kern.err kernel: [ 4720.512081] UBIFS error (ubi0:1 pid 6141): 0x801ad438: cannot read page 57 of inode 2278, error -22
Sat Dec  4 05:22:52 2021 daemon.notice netifd: radio1 (5887): Bus error (core dumped)
Sat Dec  4 05:22:52 2021 daemon.notice netifd: radio1 (5887): Interface 1 setup failed: IFUP_ERROR

/etc/config/wireless

onfig wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11g'
	option path 'pci0000:00/0000:00:01.0/0000:02:00.0'
	option htmode 'HT20'
	option cell_density '2'
	option channel '1'
	option legacy_rates '0'
	option country 'DE'

config wifi-iface 'default_radio0'
	option device 'radio0'
	option macaddr '00:xx:yy:07:ee:73'
	option network 'lan'
	option mode 'ap'
	option key '24ghzkey'
	option ssid ' 24ghz'
	option encryption 'psk2+ccmp'
	option ieee80211w '1'

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11a'
	option path 'pci0000:00/0000:00:00.0/0000:01:00.0'
	option cell_density '2'
	option legacy_rates '0'
	option htmode 'VHT80'
	option country 'DE'
	option channel 'auto'
	option txpower '17'

config wifi-iface 'default_radio1'
	option device 'radio1'
	option macaddr '00:xx:yy:07:ee:74'
	option mode 'ap'
	option key '5ghzkey'
	option encryption 'psk2+ccmp'
	option ieee80211w '1'
	option disassoc_low_ack '0'
	option network 'lan'
	option ssid ' 5ghz'

config wifi-iface 'trunk'
	option device 'radio1'
	option macaddr '00:xx:yy:07:ee:75'
	option mode 'ap'
	option ifname 'wtun'
	option network 'wtun'
	option ssid 'wifitrunk'
	option encryption 'psk2+ccmp'
	option key 'trunkkey'
	option disabled '0'
	option disassoc_low_ack '0'
	option hidden '1'
2

Those ubi (flash-) read error aren't looking good, I'd suggest to backup what you can (all mtds, all ubi partitions) and then to reflash without keeping settings. If you're lucky, that may sort it out (by writing a good image and formatting the overlay), if you aren't, the issues lay deeper (flash damage).

In case of a dual-firmware device, first switch to the other partition set and check that for consistency.

In case of the rt3200, inquire about that device specifically, you might have to rewrite some partitions to be compatible with the mainline accepted flash driver.

1 Like
3

Not sure how to backup MTDs, and no clue about ubi partitions. Can I use "Save mtdblock contents" in Luci (/cgi-bin/luci/admin/system/flash) to back up both?

df -k

Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 4352      4352         0 100% /rom
tmpfs                    61880      6192     55688  10% /tmp
/dev/ubi0_1              99088     10516     83736  11% /overlay
overlayfs:/overlay       99088     10516     83736  11% /
tmpfs                      512         0       512   0% /dev

Not sure if this Redmi AC2100 (mt7621) device is "dual firmware" or not. I do have Breed installed (is it considered "firmware"?) If I re-flash via Breed, does that fit the "reflash" you're talking about?

4

This is after flashing a intramfs image via Breed:

logread -e ubi

Sat Mar 20 23:29:50 2021 kern.notice kernel: [    7.324083] 0x000000a00000-0x000007f80000 : "ubi"
Sat Mar 20 23:29:50 2021 kern.notice kernel: [    8.886849] ubi0: attaching mtd9
Sat Mar 20 23:29:50 2021 kern.notice kernel: [    9.972663] ubi0: scanning is finished
Sat Mar 20 23:29:50 2021 kern.notice kernel: [    9.992194] ubi0: attached mtd9 (name "ubi", size 117 MiB)
Sat Mar 20 23:29:50 2021 kern.notice kernel: [    9.997697] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.004538] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.011305] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.018245] ubi0: good PEBs: 940, bad PEBs: 0, corrupted PEBs: 0
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.024223] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.031426] ubi0: max/mean erase counter: 5/2, WL threshold: 4096, image sequence number: 1614131633
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.040528] ubi0: available PEBs: 0, total reserved PEBs: 940, PEBs reserved for bad PEB handling: 20
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.049743] ubi0: background thread "ubi_bgt0d" started, PID 392
Sat Mar 20 23:29:50 2021 kern.info kernel: [   10.051663] block ubiblock0_0: created from ubi0:0(rootfs)
Sat Mar 20 23:29:50 2021 kern.notice kernel: [   10.061235] ubiblock: device ubiblock0_0 (rootfs) set to be root filesystem

Am I out of the woods yet?

5

That doesn't look bad, now you just need to get back to a sysupgrade image.

1 Like
6

Thanks. Am able to turn both WLAN radios back on, after flashing in sysupgrade image. Good to go, restoring my config backup (including gretap tunnel)?

logread |grep ubi

Mon Feb 15 15:22:49 2021 kern.notice kernel: [    2.716712] 0x000000a00000-0x000007f80000 : "ubi"
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    4.269276] ubi0: attaching mtd9
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.355065] ubi0: scanning is finished
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.374638] ubi0: attached mtd9 (name "ubi", size 117 MiB)
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.380140] ubi0: PEB size: 131072 bytes (128 KiB), LEB size: 126976 bytes
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.386982] ubi0: min./max. I/O unit sizes: 2048/2048, sub-page size 2048
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.393751] ubi0: VID header offset: 2048 (aligned 2048), data offset: 4096
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.400692] ubi0: good PEBs: 940, bad PEBs: 0, corrupted PEBs: 0
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.406669] ubi0: user volume: 2, internal volumes: 1, max. volumes count: 128
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.413869] ubi0: max/mean erase counter: 5/2, WL threshold: 4096, image sequence number: 1614131633
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.422970] ubi0: available PEBs: 0, total reserved PEBs: 940, PEBs reserved for bad PEB handling: 20
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.432179] ubi0: background thread "ubi_bgt0d" started, PID 393
Mon Feb 15 15:22:49 2021 kern.info kernel: [    5.433858] block ubiblock0_0: created from ubi0:0(rootfs)
Mon Feb 15 15:22:49 2021 kern.notice kernel: [    5.443643] ubiblock: device ubiblock0_0 (rootfs) set to be root filesystem
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.626842] UBIFS (ubi0:1): default file-system created
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.633526] UBIFS (ubi0:1): background thread "ubifs_bgt0_1" started, PID 474
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.747038] UBIFS (ubi0:1): UBIFS: mounted UBI device 0, volume 1, name "rootfs_data"
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.754871] UBIFS (ubi0:1): LEB size: 126976 bytes (124 KiB), min./max. I/O unit sizes: 2048 bytes/2048 bytes
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.764761] UBIFS (ubi0:1): FS size: 110469120 bytes (105 MiB, 870 LEBs), journal size 5586944 bytes (5 MiB, 44 LEBs)
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.775336] UBIFS (ubi0:1): reserved for root: 4952683 bytes (4836 KiB)
Mon Feb 15 15:22:49 2021 kern.notice kernel: [   10.781937] UBIFS (ubi0:1): media format: w4/r0 (latest is w5/r0), UUID 94C9ED16-620D-45BC-94D7-16C38DA4679D, small LPT model
Mon Feb 15 15:22:49 2021 user.info kernel: [   10.803698] mount_root: switching to ubifs overlay
7

Give the configs at least a glance over, there might be data corruption in there as well.

8

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.