X86 19.07.3 Bootloop

Further to my post here:
https://forum.openwrt.org/t/openwrt-19-07-3-service-release/64306/35?u=dilligaf

I took my x86 box apart and fitted a brand new msata disk to have another go with 19.07.3 and re-re-purposed a R7800 from AP to routing.

So:
combined squashfs 19.07.3 release = reboot back to grub after around 2 sec of the boot process
combined squashfs 19.07.2 release = happy

On a VM its happy.. but in my real world 19.07.3's a bit of a nightmare.

Put the old disk back in quick and returned a R7800 back to AP... I ain't going near 19.07.3 again unless anyone out there has any good ideas to proceed?

Strangely when rootfs is installed to partition, rather than bombing back to grub 19.07.3 just fails to load the kernel modules. The machine is a core i3-5010U box.

I did a back to back comparison of dmesg from an 19.07.2 ext4 vc 19.07.3 ext4 build (from image builder, but standard released images fail in the same way) in the hope that it will mean something to someone (ext4 as I'm able to recover dmesg easily from a 19.07.2 install on another partition):

19.07.2 (all working like every other Openwrt I'd tried previously)

[    0.884503] init: Console is alive
[    0.903305] kmodloader: loading kernel modules from /etc/modules-boot.d/*
[    0.906352] pps_core: LinuxPPS API ver. 1 registered
[    0.907084] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
[    0.908328] PTP clock support registered
[    0.910323] Button Hotplug driver version 0.4.1
[    0.911658] exFAT: Version 1.2.9
[    0.913618] ehci-platform: EHCI generic platform driver
[    0.915066] kmodloader: done loading kernel modules from /etc/modules-boot.d/*
[    0.926079] init: - preinit -
[    0.950898] random: jshn: uninitialized urandom read (4 bytes read)
[    0.955685] random: jshn: uninitialized urandom read (4 bytes read)
[    0.958568] random: jshn: uninitialized urandom read (4 bytes read)
[    1.030815] hub 1-1:1.0: USB hub found
[    1.032734] hub 1-1:1.0: 8 ports detected
[    1.440199] tsc: Refined TSC clocksource calibration: 2095.149 MHz
[    1.441944] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1e3349f83e9, max_idle_ns: 440795233330 ns
[    1.650287] usb 3-1: new SuperSpeed USB device number 2 using xhci_hcd
[    1.830200] usb 2-2: new full-speed USB device number 3 using xhci_hcd
[    2.015829] input: 2.4G Mouse as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/0003:1EA7:0064.0001/input/input8
[    2.017891] hid-generic 0003:1EA7:0064.0001: input,hiddev96,hidraw0: USB HID v1.10 Mouse [2.4G Mouse] on usb-0000:00:14.0-2/input0
[    2.170136] usb 2-3: new low-speed USB device number 4 using xhci_hcd
[    2.358094] input: Mini Keyboard Mini Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.0/0003:1D57:AD03.0002/input/input9
[    2.430206] hid-generic 0003:1D57:AD03.0002: input,hidraw1: USB HID v1.10 Keyboard [Mini Keyboard Mini Keyboard] on usb-0000:00:14.0-3/input0
[    2.438285] input: Mini Keyboard Mini Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.1/0003:1D57:AD03.0003/input/input10
[    2.450168] clocksource: Switched to clocksource tsc
[    2.510210] hid-generic 0003:1D57:AD03.0003: input,hidraw2: USB HID v1.10 Mouse [Mini Keyboard Mini Keyboard] on usb-0000:00:14.0-3/input1
[    2.660131] usb 2-4: new low-speed USB device number 5 using xhci_hcd
[    2.848413] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4:1.0/0003:1A2C:0B2A.0004/input/input11
[    2.920201] hid-generic 0003:1A2C:0B2A.0004: input,hidraw3: USB HID v1.10 Keyboard [SEM USB Keyboard] on usb-0000:00:14.0-4/input0
[    2.927978] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4:1.1/0003:1A2C:0B2A.0005/input/input12
[    3.000210] hid-generic 0003:1A2C:0B2A.0005: input,hidraw4: USB HID v1.10 Device [SEM USB Keyboard] on usb-0000:00:14.0-4/input1
[    3.993099] mount_root: mounting /dev/root
[    3.995873] EXT4-fs (sdb4): re-mounted. Opts: (null)
[    3.997021] mount_root: loading kmods from internal overlay
[    4.000583] kmodloader: loading kernel modules from //etc/modules-boot.d/*
[    4.001925] kmodloader: done loading kernel modules from //etc/modules-boot.d/*
[    4.027260] block: attempting to load /etc/config/fstab
[    4.028990] block: extroot: not configured

And then 19.07.3 (like for like build - gaps are from notepad++ file compare):

[    0.865009] init: Console is alive
																			
													   
																									   
										   
												  
								   
														  
																				 
[    0.876443] init: - preinit -
[    0.899321] random: jshn: uninitialized urandom read (4 bytes read)
[    0.903747] random: jshn: uninitialized urandom read (4 bytes read)
[    0.906028] random: jshn: uninitialized urandom read (4 bytes read)
[    1.030855] hub 1-1:1.0: USB hub found
[    1.032612] hub 1-1:1.0: 8 ports detected
[    1.430106] tsc: Refined TSC clocksource calibration: 2095.149 MHz
[    1.431843] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x1e3349f83e9, max_idle_ns: 440795233330 ns
[    1.650196] usb 3-1: new SuperSpeed USB device number 2 using xhci_hcd
[    1.830109] usb 2-2: new full-speed USB device number 3 using xhci_hcd
[    2.015187] input: 2.4G Mouse as /devices/pci0000:00/0000:00:14.0/usb2/2-2/2-2:1.0/0003:1EA7:0064.0001/input/input8
[    2.017262] hid-generic 0003:1EA7:0064.0001: input,hiddev96,hidraw0: USB HID v1.10 Mouse [2.4G Mouse] on usb-0000:00:14.0-2/input0
[    2.170044] usb 2-3: new low-speed USB device number 4 using xhci_hcd
[    2.358222] input: Mini Keyboard Mini Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.0/0003:1D57:AD03.0002/input/input9
[    2.430107] hid-generic 0003:1D57:AD03.0002: input,hidraw1: USB HID v1.10 Keyboard [Mini Keyboard Mini Keyboard] on usb-0000:00:14.0-3/input0
[    2.437972] input: Mini Keyboard Mini Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-3/2-3:1.1/0003:1D57:AD03.0003/input/input10
[    2.444145] clocksource: Switched to clocksource tsc
[    2.510113] hid-generic 0003:1D57:AD03.0003: input,hidraw2: USB HID v1.10 Mouse [Mini Keyboard Mini Keyboard] on usb-0000:00:14.0-3/input1
[    2.660040] usb 2-4: new low-speed USB device number 5 using xhci_hcd
[    2.848119] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4:1.0/0003:1A2C:0B2A.0004/input/input11
[    2.920110] hid-generic 0003:1A2C:0B2A.0004: input,hidraw3: USB HID v1.10 Keyboard [SEM USB Keyboard] on usb-0000:00:14.0-4/input0
[    2.927659] input: SEM USB Keyboard as /devices/pci0000:00/0000:00:14.0/usb2/2-4/2-4:1.1/0003:1A2C:0B2A.0005/input/input12
[    3.000113] hid-generic 0003:1A2C:0B2A.0005: input,hidraw4: USB HID v1.10 Device [SEM USB Keyboard] on usb-0000:00:14.0-4/input1
[    3.943220] mount_root: mounting /dev/root
[    3.947212] EXT4-fs (sdb4): re-mounted. Opts: (null)
[    3.949404] mount_root: loading kmods from internal overlay
[    3.952709] mount_root: failed to launch kmodloader from internal overlay
																				  
[    3.977261] block: attempting to load /etc/config/fstab
[    3.978565] block: extroot: not configured

I'm taking it that this is a bug. Though not sure why it's not reported by anyone else here.
To me it is what it says it is - none of the kernel modules are getting loaded but I've no idea why that should be or if there's anything I can do to fix that? I'm vexed as changes should be minor in a service release.

If anyone who understands anything about the boot process could provide information or help I'd really appreciate it. Or even just advise that I should report it as a bug.

Is this a custom build?

Edit: you're also mentioning another partition. You have multiple OpenWrt installs on the same physical disk? If yes, make sure that grub is booting the right kernel and that the kernel is mounting the correct rootfs. Iirc it is selected by a semi-random PARTUUID value.

Only in the sense that I've added my config files and additional standard packages to the image builder from the release folder - it just helps me transfer the dmesg onto another internal disk in the event of success.

The result is the same (kernel modules fail to load) if I download and the standard builds from the 19.07.2/19.07.3 release folder.

My grub entries are:

menuentry "OpenWrt2" {
	linux /boot/vmlinuz root=PARTUUID=57113300-03 rootfstype=squashfs rootwait console=tty0 console=ttyS0,115200n8 noinitrd
}

…
menuentry "OpenWrtExt4" {
	linux /boot/vmlinuz root=PARTUUID=57113300-04 rootfstype=ext4 rootwait console=tty0 console=ttyS0,115200n8 noinitrd
}

That looks odd. So you're using the same kernel to boot 19.07.2 and 19.07.3 root filesystems? That will not work as the /lib/modules/ folder is kernel version specific and the Kernel version differs between 19.07.2 and 19.07.3

Yeah.. the penny just dropped. Thanks!

Am I good to just rename the 19.07.3 kernel and call it from grub?

Yep, that should work just fine.

Thanks again. All multibooting happy. Nice one.

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.