Overriding KERNEL_PATCHVER on a per-device basis?

I just recently dug out my old GL.iNet AR-300M and started playing with it.

There is currently support for this device using NAND in master, HOWEVER, as documented in http://lists.infradead.org/pipermail/openwrt-devel/2019-January/015632.html - this "support" is broken with 4.14 (which is the current specified KERNEL_PATCHVER for ath79 devices) and results in a device that will not boot

I tried manually changing KERNEL_PATCHVER in https://github.com/openwrt/openwrt/blob/master/target/linux/ath79/Makefile to 4.19 and the built image boots and seems to work quite well.

It seems to me that it's a bad idea to have a selectable item in menuconfig that is guaranteed to create a nonbootable image. I assume this is why Jeff K submitted that patch to openwrt-devel (no one bothered to respond to his submission with comments) - is it possible to override KERNEL_PATCHVER for all ath79 NAND targets (currently only the AR300M - but I'm looking into implementing AR750S NAND support based on the AR300M work done already)? If so - which Makefile would this ideally be done in?

The ath79 target recently added patches for 4.19. Swiitching the ath79 targets on your local builder to Linux 4.19 can be accomplished by applying (on master)

diff --git a/target/linux/ath79/Makefile b/target/linux/ath79/Makefile
index 82f6317ac6..cfdf8c87e5 100644
--- a/target/linux/ath79/Makefile
+++ b/target/linux/ath79/Makefile
@@ -8,7 +8,7 @@ SUBTARGETS:=generic nand tiny
 include $(INCLUDE_DIR)/target.mk

Yup, as I mentioned, manually changing it there works - however it would be nice if a solution could be integrated into master that only switches over targets that require 4.19 for now, leaving 4.14 as is until the OpenWRT team finishes their 19.???? release efforts.

The OpenWrt build system is a single kernel for all devices on a given target. So, no, that is not available as a feature of the build system at this time, as I understand it.

(Would you poke me if you get the GL.iNet AR750S running off NAND? That's on my overly long short list.)

Will do. I recently sent you (I'm assuming you are Jeff K) an email - not sure if you found this forum post because I linked it.

For the AR750S, what I'm not sure about is how to handle the setup for a device that stores kernel on one device and rootfs on another, see gl_ar750s_mtdlayout in https://github.com/gl-inet/openwrt/blob/develop/target/linux/ar71xx/image/legacy.mk#L277 - do you know of any devices with a similar multi-device layout I could look at for inspiration?

edit: https://openwrt.org/docs/guide-developer/defining-firmware-partitions doesn't seem to cover this particular scenario

Yes, one and the same person. I'll check my mail

I can't think of any off-hand where they are on different devices. I would think/hope that the boot loader would be able to load the kernel from the NOR, and then the kernel command line could specify where the root file system resides. As long as the kernel had the UBIFS drivers built in, init should be able to mount off NAND. The GL.iNet U-Boot is one of the best OEM boot loaders I've run across so changing the boot args I would imagine would be straightforward, if needed at all.

Edit: I put the AR750S on hold until 4.19 was ported to ath79. I've been working on an ipq4019-based device that I thought was going to be an "easy port" that has led me down into the depths of current Linux development of DSA architecture and implementation.

Looks like WNDR4300 v1 might have some hints. In gl.inet's own sources, they have "ubinize-.ini" for their NAND devices. The WNDR4300 v1 has this in the ar71xx/image directory of OpenWRT master, but not anywhere in ath79 - in fact the device does not appear to have been migrated at all, so that probably can't be used as a reference. :frowning:

WNDR4300 v2 is an SPINAND device similar to the GL.iNet devices and has no support in OpenWRT - people who want this router to work might find inspiration in the recent 4.19 spinand work

I just discovered that despite having 128M of NAND, something in the AR300M chokes on images larger than 16M even for NAND... Which kind of defeats the purpose of NAND until I can figure out why. :frowning:

Are you flashing from NOR-based to NAND-based? If so, the sysupgrade tooling probably can't cross the versions. U-Boot should work for the NAND-based image, no matter the current OpenWrt firmware.

I have vague memories that the NAND-based firmware needs to be named *.tar (edit -- or something like that) for U-Boot to recognize it as destined for the NAND. file some-build.bin should be able to confirm that you've got a tar-style upload in hand, even if OpenWrt names it *.bin

So what I've seen so far is:
NAND-based OpenWRT builds won't flash a sysupgrade tar without adding append-metadata or forcing (I'm rebuilding with that now)
If you use append-metadata, a recent build will still refuse to flash because the current board type is "unknown" - looks like something isn't being set properly on current NAND builds
GL.iNet's uboot webfailsafe mode has a small bug where it only uses the NOR size when deciding whether to accept a firmware file. So it can't flash large NAND images.

Doing a TFTP flash is going to be difficult, since the latest released uboot image for the AR300M is missing the "nlf" script referenced in the documentation for loading NAND firmware from the serial console. There's a lot of discussion on the GL.iNet forums where one of their developers posted an updated uboot image, unfortunately:

  1. They appear to have not bothered to comply with the GPL for that image, as no changes were made to their published source to fix the issues the binary supposedly fixed
    1a) Even if they had, apparently you can't build their uboot images with Ubuntu 18.04 and newer, only 16.04
  2. The "fixed" image in question appears to have had the attachment deleted at some point


I think I'm going to start with <16M NAND builds and trying to work out the kinks in sysupgrade - then start flashing larger builds via sysupgrade once it's working reliably

1 Like

Might also be able to flash from an NAND-enabled initramfs image loaded over TFTP to RAM and booted from RAM

BTW, drop them a line on the U-Boot source. They are really on top of things and strive to do things right from all the interactions I have had with them.

I'll ping them sometime next week. The weekend and timezone change probably don't help - but there have been multiple requests in the forums in the past 1-2 weeks for them to repost the "fixed" AR300M uboot with no response so far.

I also noticed something in the uboot source (see CONFIG_MTDPARTS_DEFAULT in the uboot source) that sort of implies that the rootfs can't grow to more than 20M even on NAND - so it looks like "baking in" a lot of packages (as opposed to installing them after the fact, which is problematic due to firewalling in some of my use cases) might be an issue at least on AR300M.

For the most part my primary interest is the 750S - but I want to work out the kinks in the AR300M first since those will also be needed on the 750S.

Once U-Boot has launched the kernel, it's notion of MTD parts is done. Yes the kernel might be configured in some cases to accept them as "suggestions" from the command line, but it certainly doesn't have to. I think as long as U-Boot can find and load the kernel to RAM, all is good.

Yeah, I'll definitely take another look in a bit.

I'm going to wait a bit to see if GL responds to the requests for them to repost their updated uboot - being able to switch back/forth between NOR and NAND on the fly would help a bit. The current uboot version I have is pretty easy to knock into a state that requires serial intervention. I am also going to look into printing a case for the 300M that gives me access to the serial port without having a completely bare board floating around.

Also, I did some digging around/reading - target/linux/ath79/base-files/lib/upgrade/platform.sh definitely needs work for NAND targets.

Not sure if I'll take a crack at that tomorrow or wait for some progress on the uboot side of things.