Problems with X86 non-squashfs combined img, Wrong Approach?

Sorry, but I've tried this every time I've upgraded since 17 and I can't get it to work. I don't know what I'm doing wrong. When I build images with ImageBuilder, I've been able to tweak a few things, like the root FS size. But, I keep thinking things would be simpler if I could just have a real ext4 FS. So, I keep trying to use the non-squashfs image. But, it never works. I always have two problems

  1. it boot readonly root fs, which I don't think could work, since I still need to configure it (restore my backup config).
  2. it doesn't have any network. all the regular firmware docs say it will boot and start http on 192.186.1.1, but none of the interfaces are even started when I boot the image.

Now, maybe this is just the wrong approach. I keep trying this because of two other problems I've had:

  1. I can't seem to get sysupgrade to work with the x86 images.
  2. I can't seem to get the FILES= option to work so that my new, custom image has my configuration on it. Every time i've tried, the image has some problem, like it will not boot (IIRC) or maybe the same thing where there's no network and the configuration I include doesn't get into the image.

But, I guess having a writable root FS doesn't really help much. It's not really going to make upgrading openWRT any easier, right?

So, what should I do?

A) Is there any use to getting the non-squashfs, readonly FS to work?

B) Any idea what I could be doing wrong with FILES=? That seems like it would be a big help if I could get that to work. (I asked about this once before.)

C) Any idea why sysupgrade doesn't seem to work for me on this kind of installation? I don't remember what happened, but I think I had to blast a new image over everything the last time I tried it.

D) I saw an old thread that discussed using a separate partition and then just flashing a boot drive with new versions. That's basically what I'm trying to do. I have other storage on the system and, when I get my config restored, it is then accessible. Is there a way that the configuration files can live on the permanent, read/write partition and have the image I make with ImageBuilder pull from there or something? Again, if I could make FILES=work, I'm assuming that would solve everything.

Does the official image work?

The issues you're having seem quite odd.

1 Like

I'm with @lleachii, this is very odd. I've been using x86 on various devices for years and don't have any of the issues that you're seeing.

  • My "production" router is x86 (AMD Jaguar) BIOS squashfs at default rootfs-size on 23.05.3, updated whenever there's a new stable release..
  • My "dev" router is x86 VM EFI ext4 at 128MB rootfs-size, with (usually) several weeks old SNAPSHOT.
  • My "test" router is x86 (Intel N5105) EFI squashfs 256 MB rootfs size, SNAPSHOT updated 2-10x per day. Sometimes I'll switch it to ext4 or rootfs 512 MB or whatever, and quite often I'll test the "FILES=" uci-defaults on test upgrades.

The only issue I recall in the past few years was when I made the rootfs too small (100 MB? something like that) on good old "test" and the sysupgrade didn't restore the backup automatically (but it is a test router, so no big deal).

For imagebuilder testing, I'll do something like this.

For FILES='path to root of extra files' , say you want to add a new uci defaults script and a data file to /usr/lib, copy files to a directory structure emulating the target:

  files=/home/efahl/somedir/files
  mkdir -p $files/etc/uci-defaults
  mkdir -p $files/usr/lib
  cp startup $files/etc/uci-defaults/03-startup
  cp data    $files/usr/lib/my_data.txt

Then my command line looks like:

make image \
  PROFILE="generic" \
  ROOTFS_PARTSIZE=256 \
  FILES="$files" \
  PACKAGES="conntrack nmap vim-full blah blah blah"

I think so, but I haven't tried it in a long time. I can't seem to get ImageBuilder to work, so maybe that makes sense to try. I just haven't considered it because I need so many packages that I don't think weren't included when I first started using openwrt that it just make sense to put them on with ImageBuilder. But, that assumes I'm using it right, and maybe I'm not. Also, one time I couldn't install needed packages because there wasn't enough space. At 110 MB, I don't think I could install the pkgs I need. But, lets assume the official, generic, x86 image works.

That means I'm messing up the ImageBuilder process, then, right?

  1. So, you're saying that sysupgrade should work as normal. It's not any different on x86. Okay, I wasn't sure.
  2. Also, you're saying you don't have any problems with FILES=. But, I'm sure I had problems with it. Here's what I would like to do. I just want to point files=/path of unziped backup config/ . Does that make sense? That's what I tried before and it didn't work.
  3. I've not heard of ROOTFS_PARTSIZE= option before. Where is that documented? I normally adjust CONFIG_TARGET_ROOTFS_PARTSIZE in .config.
  4. I see some references to make menuconfig with ImageBuilder in some documents, but there is no such target. What am I missing?
  5. I normally just run make image PACKAGES="..." . Is that wrong? Do I always need to specify a PROFILE? Maybe that's what I'm doing wrong.
  6. When you use the non-squashfs root FS, does it boot readonly or not? I'm confused on what I should be seeing.
1 Like
  1. Yes, I've never seen anything different when installing any of ext4, squashfs or swapping back and forth between them. It just works.

  2. I've never tried that, but if you try to restore /etc/config/ from FILES= make sure you install it with sysupgrade -n or I'm pretty sure most of it will be overwritten by the sysupgrades -b/-r save/restore process.

  3. Built-in help (probably something on the wiki, but I'm too lazy to look :grin:):

~/image_builder/openwrt-imagebuilder-x86-64.Linux-x86_64$ make help
... snip ...
image:
        By default 'make image' will create an image with the default
        target profile and package set. You can use the following parameters
        to change that:

        make image PROFILE="<profilename>" # override the default target profile
        make image PACKAGES="<pkg1> [<pkg2> [<pkg3> ...]]" # include extra packages
        make image FILES="<path>" # include extra files from <path>
        make image BIN_DIR="<path>" # alternative output directory for the images
        make image EXTRA_IMAGE_NAME="<string>" # Add this to the output image filename (sanitized)
        make image DISABLED_SERVICES="<svc1> [<svc2> [<svc3> ..]]" # Which services in /etc/init.d/ should be disabled
        make image ADD_LOCAL_KEY=1 # store locally generated signing key in built images
        make image ROOTFS_PARTSIZE="<size>" # override the default rootfs partition size in MegaBytes
... more snippage ...
  1. There's no make menuconfig once you're in the imagebuilder itself. You'd use menuconfig when using a source build to create the imagebuilder. (But then why would one bother with imagebuilders? Just make the image you want directly... Imagebuilders are just a convenient means for packaging up various build artifacts so end users don't have to set up a full build environment.)

  2. I would think that the default for x86/64 would be generic, but I always am explicit. Yes, it is possible that this may be a source of your issues.

  3. I've only ever had them boot up as rw file systems on root.

On squashfs, there is the read-only /rom partition of course, but everything is exposed rw in /overlay and mapped to "where it should be" by default (i.e., /etc is really rw /overlay/upper/etc/config).

$ mount | grep ' / '
overlayfs:/overlay on / type overlay (rw,noatime,lowerdir=/,upperdir=/overlay/upper,workdir=/overlay/work,uuid=on,xino=off)

On ext4, it's all just rw in its proper place.

$ mount | grep ' / '
/dev/root on / type ext4 (rw,noatime)

Hmm, you are using the FILES= option, but you're not using it to build an image that already has the /etc/config files on it. I can't think of a more useful way to use this option. Why aren't you using this option to include your config files? I must be missing something.

Also, I have no idea what you mean about sysupgrade. what is -n? Is that the full upgrade of everything? Oh, okay, I think I understand now.

But, if you include your actual config with FILES=, then when you do sysupgrade -n wouldn't you get what you want without having to restore the config? That just make so much sense to me, I'm having a hard time thinking of why you wouldn't want to do that. Oh, if the config is being restored from ROM on each boot, I guess that would overwrite/overlay it. Is that what you mean? Yeah, sure, okay, that makes sense, but it's still a better first config than the default one. Right? I mean, if something happens to the ROM or you're using non-squashfs, then building your image with the config you want makes sense right?

I have a config backup, which I got through LuCI, but I assume it is just sysupgrade -b on the backend. In the past, the only why I have ever been able to upgrade is by flashing a new image, direct connect ethernet, connect to http server, login, then use restore from LuCI, which I assume is actually sysupgrade -r on the backend.

Those are the files I'm trying to include with FILES=. Is there any reason not to do this? Is there a better way, assuming I can't use the normal method of upgrading the whole system and I need to overwrite it?

Hmm, I suppose I could include the config backup .gz file in my image, or bring over with USB, and then use the console to run sysupgrade -r manually. That might be easier than having to boot it on the default config, manually configure my labptop NIC... Okay, yeah, I guess that makes more sense than what I've been doing.

Right, my only "real" use of imagebuilder is to create images that have a bunch of extra PACKAGES= in them. My use of FILES= has just been experimental to make sure I know how it works.

Typical sequence is

build-host$ make image blah blah blah
build-host$ scp bin/targets/x86/64/openwrt-x86-64-generic-ext4-combined-efi.img.gz router:/tmp/firmware.bin

Now on to the installation

build-host$ ssh router

router$ sysupgrade --test /tmp/firmware.bin
Mon Jun 24 16:37:09 PDT 2024 upgrade: Image metadata not present
Mon Jun 24 16:37:09 PDT 2024 upgrade: Reading partition table from bootdisk...
Mon Jun 24 16:37:09 PDT 2024 upgrade: Extract boot sector from the image
Mon Jun 24 16:37:09 PDT 2024 upgrade: Reading partition table from image...

router$ sysupgrade /tmp/firmware.bin
... installs and reboots router ...

That final sysupgrade implicitly does a sysupgrade --create-backup just prior to installing the image (thus saving /etc/config/* and a bunch of other package-specific stuff; run sysupgrade -l if you're curious), then upon reboot it performs a sysupgrade --restore-backup /tmp/backup.tgz (or wherever it is). This means that all of my router-specific config is carried over; the sole reason for using the imagebuilder was to get the packages into the image.

Use of sysupgrade -n /tmp/firmware.bin would explicitly not perform the backup/restore, and hence anything in /etc/config and many other places would not be overwritten on reboot. It would be as if I had installed the basic firmware, then manually installed all the packages from scratch and not done any configuration at all.

Are you perchance running SNAPSHOT? You might be interested in my router-based upgrade tool, owut. It does a lot of this stuff from just a single command (and I'm looking for testers :grinning:).

Okay, you are way ahead of me, but you have the same use case for using ImageBuilder. I had a lot of extra pkgs I want. So, I think I'm doing the right thing. And I want a big root partition because I even want to try running docker and that takes a ton of stuff.

I'm not running SNAPSHOT. I just want to run OpenWRT with some local services. Hmm, your tool looks pretty cool.

Hmm, okay, looks like I've been typing PACKAGE=... that's clearly a problem, but not sure it explains my recent problems.

Is see that the "official" x86 image has many more files in /etc than the one I created with ImageBuilder. So, clearly I didn't do it right this last time. I just don't do this often enough to remember all the stuff I learned when I figured it out the last time.

Thanks for your help. I think I know what to try next, at least.

I'm pretty dead set on moving to non-squashfs. At least then I can change the partition size if needed. Although that seems like not much a good reason. And, I'm assuming you cannot sysupgrade from the squashfs layout to the non-squashfs layout. Right?

Actually, it's surprisingly easy. Just pick the image with ext4 in the name instead of squashfs and install that. Bingo, you're now running the non-squash fs...

Explanation: these are images that overwrite the whole drive, so no partition tables or info regarding the file system is retained when you install them.

Here what I got after running imagebuilder, choose the combined + efi or non-efi/bios version that suits your hardware:

image_builder/openwrt-imagebuilder-x86-64.Linux-x86_64$ ll bin/targets/x86/64/
Permissions Size User      Date Modified    Name
.rw-r--r--@  12M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-ext4-combined-efi.img.gz
.rw-r--r--@  12M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-ext4-combined.img.gz
.rw-r--r--@ 5.1M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-ext4-rootfs.img.gz
.rw-r--r--@  12M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-squashfs-combined-efi.img.gz
.rw-r--r--@  11M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-squashfs-combined.img.gz
.rw-r--r--@ 4.2M efahlgren 2024-05-12 10:10 openwrt-x86-64-generic-squashfs-rootfs.img.gz

In case you aren't, sure of BIOS vs EFI...
On old AMD Jaguar:

$ [ -d /sys/firmware/efi ] && echo "It's EFI" || echo "It's BIOS"
It's BIOS

On new Intel N5105:

$ [ -d /sys/firmware/efi ] && echo "It's EFI" || echo "It's BIOS"
It's EFI
1 Like

https://openwrt.org/docs/guide-developer/uci-defaults

The Firmware Selector can also be used to build images and add UCI defaults.

https://firmware-selector.openwrt.org

Right, but he also wants to change root fs size, which isn't supported by FS...

1 Like

:man_facepalming: Oops, I forgot that part.

Hmm, okay, I'm not exactly sure what you are getting at with that. Are you saying I should use uci-defaults script INSTEAD of trying to overwrite config files with live config files? Or are you just say there is another way that people like to use.

I see the value of this approach. If I had learned OpenWRT from the uci CLI 10 years ago, that approach likely would have occurred to me. But, I have no idea how to rebuild the configuration I have from the CLI. What I do have is a .gz backup bundle that contains 10 years of customized config files. I've built what I want from a combination of CLI and LuCI web controls. The only way I know how to install it on a new system or upgraded system is by using these files, somehow. I hope that explains why I'm so keen on using FILES=.

Okay. That's great. I guess I assumed that wouldn't work. Well, actually, I'm pretty sure I tried it once and it didn't work. But, whatever. It's worth another try.

I've built a new image with quite a few new tricks I've learned from this thread. Thanks all. I'll try it later tonight.

Of all the things I've reported that didn't work for me. Using FILES= to include all my backup config files is the thing I've tested the most and I'm most confident it didn't work. But, unless someone tells me they know why that might not work, I'll give it another try. If it doesn't work, I guess I'm no worse off and I can start over, which is what I usually end up having to do with the upgrade.

Where does it put the backup that it can restore? Isn't it overwriting the boot disk and root FS?

Well, I think it actually worked, but I panicked when ssh shutdown and thought it hung while booting, when in fact, it looks like sysupgrade was just writing the new image. Argh! So close. I messed up that drive, but it booted fine off the SDcard I had ready to go, no drama. FILES= finally worked like I had hopped it would.

I guess I had no expectation for how long it should take or what to expect while it did the update. I now realize that it was still passing traffic, so I shouldn't have worried, yet. In any case, I guess sysupgrade and FILES= were working and I can try it again next time. It worked well enough, though, that I think my notes will save me hours of work next time. And, I'm not dreading the next major update, so that's a good feeling.

Thanks for your help.

I'm not sure how to close this thread, but I'll note that I used PROFILE=generic, which I didn't think was actually necessary, and I fixed an error in the PACKAGES= option. But, I think the rest of my procedure was valid.

1 Like

Uh oh. / is still not writable. Dang. I used the non-squashfs image and I still have this issue. Okay, I still need help.

This is outside my area of comfort and I'm hoping we can get a sysadmin type to chime in here, but it sounds like a post-boot mount problem.

ssh into your router and show us what's in the two fstab files.

cat /etc/fstab
cat /etc/config/fstab