OpenWrt 19.07 on devices with small kernel partition

nybash · July 9, 2020, 4:29pm

I was surprised when image for my AP147_10 (ar71xx) was not built because of kernel partition is less than lzma image.

(ImageBuilder's output)

build_dir/target-mips_24kc_musl/linux-ar71xx_generic/tmp/vmlinux-ap147-010.uImage is too big (> 1507328 bytes)

Meanwhile It has a SPI flash with 16Mb and 128Mb RAM!

Then I've downloaded git sources and tried to build a shrank kernel image by make kernel_menuconfig and cut down unusable ethernet drivers:

But, it seems that kernel size is the same as default.

ls -la build_dir/target-mips_24kc_musl/linux-ar71xx_generic/tmp/
total 8196
drwxr-xr-x  2 root root    4096 Jul  9 14:22 .
drwxr-xr-x 14 root root    4096 Jul  9 14:22 ..
-rwxr-xr-x  1 root root 5172212 Jul  9 14:22 vmlinux-ap147-010
-rw-r--r--  1 root root 1604271 Jul  9 14:22 vmlinux-ap147-010.bin.lzma
-rw-r--r--  1 root root 1604335 Jul  9 14:22 vmlinux-ap147-010.uImage

How to be?

apccv · July 11, 2020, 5:42pm

I'm not sure I know what I'm doing... but I would try playing with the layout definition:

ap147_mtdlayout=mtdparts=spi0.0:256k(u-boot)ro,64k(u-boot-env)ro,14528k(rootfs),1472k(kernel),64k(art)ro,16000k@0x50000(firmware)

in target/linux/ar71xx/image/legacy.mk . Just be careful not to nuke the ART partition. Make a backup of it to begin with. Maybe make rootfs smaller to make space for a larger kernel?

Maybe poke around the device definition and try to find where the kernel image limit is set.

Might be a waste of time if the problem is the bootloader cannot handle a larger kernel than what's defined already... but can't hurt trying (as long as you have an ART backup and u-boot has some recovery mechanism to recover the brick).

nybash · July 12, 2020, 6:34am

As I know, the art partition on my devices is empty, because mac address equals "00:02:03:04:05:06", I bought it as a pure board without box.

I thought about playing with layout, but will need to seize that one from work space and connect through UART to experiments.

Anyway, vocore2 builder easily expanded kernel partition. It is possible that the AP147_010's bootloader has hardcoded layout

nybash · July 12, 2020, 6:42am

Why can't I throw away unnecessary hardware from kernel to shrink the size? I could not find something specific about kernel_menuconfig, whatever I did, lzma image still the same size.

nybash · July 12, 2020, 1:19pm

The image has gotten, but not tested yet:
ap147_mtdlayout=mtdparts=spi0.0:256k(u-boot)ro,64k(u-boot-env)ro,14328k(rootfs),1672k(kernel),64k(art)ro,16000k@0x50000(firmware)

I subtracted 200Kb from rootfs and added that to kernel partition. Still don't know correctly is... I think it can cause problems with rootfs_data partition at least... but maybe u-boot will not find kernel.

svanheule · July 13, 2020, 3:39pm

Seeing that there is a u-boot-env partition, would it be an option to change the entry-point after the bootloader? If you can make it boot from 0x50000 instead of 0xe8000, you wouldn't have to care as much about not running into the art partition and you could just use a dynamic firmware partition split.

nybash · July 14, 2020, 5:30am

What I have scratched from u-boot-env partition:

# fw_printenv 
bootargs=console=ttyS0,115200 root=31:02 rootfstype=jffs2 init=/sbin/init mtdparts=ath-nor0:256k(u-boot),64k(u-boot-env),14528k(rootfs),1408k(uImage),64k(mib0),64k(ART)
bootdelay=2
baudrate=115200
ethaddr=0x00:0xaa:0xbb:0xcc:0xdd:0xee
dir=
lu=tftp 0x80060000 ${dir}u-boot.bin&&erase 0x9f000000 +$filesize&&cp.b $fileaddr 0x9f000000 $filesize
lf=tftp 0x80060000 ${dir}ap143${bc}-jffs2&&erase 0x9f050000 +0xE30000&&cp.b $fileaddr 0x9f050000 $filesize
lk=tftp 0x80060000 ${dir}vmlinux${bc}.lzma.uImage&&erase 0x9fE80000 +$filesize&&cp.b $fileaddr 0x9fE80000 $filesize
bc=AP147-16M
filesize=28780
fileaddr=80060000
lok=tftp 0x80060000 openwrt-ar71xx-generic-${bc}-kernel.bin && erase 0x9f680000 +${filesize} && cp.b $fileaddr 0x9f680000 0x160000
lqsdk=run lof && run lok
bootcmd=bootm 0x9fE80000
lof=tftp 0x80060000 openwrt-ar71xx-generic-${bc}-squashfs-sysupgrade.bin && erase 0x9f050000 +${filesize} && cp.b $fileaddr 0x9f050000 $filesize
stdin=serial
stdout=serial
stderr=serial
ethact=eth0
ipaddr=192.168.2.2
serverip=192.168.2.1

It confuses me even more, because the kernel partition size is 1408k

svanheule · July 14, 2020, 7:37am

My guess is that the bootargs from u-boot are patched out in the OpenWrt kernel. To be sure, you can check /proc/cmdline in OpenWrt to see what bootargs were actually used.

I don't really understand how ar71xx/legacy.mk works, but maybe changing the partition layout to the following will be sufficient to generate an image with kernel:rootfs instead of rootfs:kernel:
ap147_mtdlayout=mtdparts=spi0.0:256k(u-boot)ro,64k(u-boot-env)ro,16000k(firmware),64k(art)ro

If you can then permanently modify u-boot to boot from the start of the firmware partition, you don't have to worry about the kernel size any more.

setenv bootcmd bootm 0x50000
saveenv

nybash · July 19, 2020, 12:02pm

One question before I'll continue: why is the image weight 17970432 bytes instead 16262165 which I had before? Is it normal?

nybash · July 19, 2020, 12:13pm

It needs to specify. What binwalk shows me on old image which works now:

And new, generated by "16000k(firmware)"

I think it does not fit

nybash · July 19, 2020, 3:19pm

So, I turned back to my previous idea and got a positive result!
It took times to calculate "magic number" to boot kernel. I opened old sysupgrade image in hex editor, then found the address where kernel starts, then did the same for new image and then subtracted it from old address, then the result subtracted from old magic number.
Thus:

00E30000 (old kernel address) - 00DFE000 (new kernel address) = 32000
9fE80000 (old magic number) - 32000 = 9FE4E000

svanheule · July 19, 2020, 5:11pm

It seems that the build system uses rootfs or firmware partition size and always appends a uImage with the kernel. In that case, my suggestion by itself wont be enough and you have to find a way to place the uImage before the rootfs.

That 'magic' number is the address where the flash is mapped to. Address 0x9f000000 refers to the first byte of the flash. 0x9fe80000 is just the address of the kernel image then (0xe80000 = 256kiB+64kiB+14528kiB).

If you change the partition sizes, note that it might be best to stick to multiples of 64k.

nybash · July 19, 2020, 6:02pm

Ok, but why and what does it threaten in the future?

svanheule · July 19, 2020, 6:17pm

I don't know if you've tried running with a modified firmware layout yet, but e.g. the mtd splitter that dynamically creates the rootfs-data overlay rounds partition sizes to an erase block. If the end of your rootfs partition doesn't align with an EB, then your kernel may or may not get corrupted.

nybash · July 19, 2020, 6:17pm

Wow! I have never calculated hex before: (1024*256=262144=0x40000) + (1024*64=65536=0x10000) + (1024*14528=14876672=0xE30000) = 15204352 or 0xE80000
Just leave it here to come back if I'll forget again

nybash · July 19, 2020, 6:28pm

Yes, I ran, it works.

rohannathi · August 13, 2020, 9:52am

Even I am facing the same issue with 19.07 branch for ap147-10, being a newbie to Openwrt platform, can you please tell the final fix to generate squashfs.bin image.

nybash · August 13, 2020, 1:14pm

There is no final fix, I've just done it for myself.
Ok, I'll tell you the algorithm what I did:

get git repository
Edit file target/linux/ar71xx/image/legacy.mk and replace ap147_mtdlayout with
ap147_mtdlayout=mtdparts=spi0.0:256k(u-boot)ro,64k(u-boot-env)ro,14328k(rootfs),1672k(kernel),64k(art)ro,16000k@0x50000(firmware)
make menuconfig and choose the platform.
make
you need to connect to your device via UART and use any terminal emulator (for example putty)
upload image with uboot menu and tftp (you can upload it via web or sysupgrade, but the device will not boot), I recommend to learn to use tftp if something went wrong and you'll want to get back an old image:
tftpboot 0x80060000 openwrt-19.07.3-ar71xx-generic-ap147-010-squashfs-sysupgrade.bin && erase 0x9f050000 +${filesize} && cp.b $fileaddr 0x9f050000 $filesize
change magic number to new address:
setenv bootcmd bootm 0x9FE4E000
saveenv

I could help you to build the image, but you anyway need to get in to uboot menu and change magic number. ( item 7).

rohannathi · August 17, 2020, 9:04am

Hi thanks, I tried this workaround, but still could not get the openwrt-19.07-ar71xx-generic-ap147-010-squashfs-sysupgrade.bin image, Is there any more files to modified

How did calculate the magic number, is there any way to update this magic number, before the build in the uboot package.

nybash · August 20, 2020, 7:08am

Hmmm... As I understand, you need to update u-boot-env partition as well, but it is in read-only more, I did not think about it.
The second way is to shrink the kernel size by cut out unneeded options. This what I tried to do at first, but fail.