Do sysupgrades wipe & reinstall the whole disk including partition table, or just partition contents? (HDD, NAS usage)

Installing and Using OpenWrt
1

Hello,

I installed OpenWRT onto a Mybook Live NAS which - after the boot & root partitions - also has a large data partiton.
Of course, this data partition must persist after sysupgrades. However, I had to use a GPT partition table (using gdisk --mbrtogpt once) to be able to create this (8TB) partition at all, because fdisk only supports partitions up to 2TB. This means that if a sysupgrade were to rewrite the partition table (back to MBR format), my data partition would not be accessible after that.

My question is, does a sysupgrade wipe & reinstall

  • the entire disk including partition table (this would be bad) or
  • just the contents of the kernel and root partitions?

I read "Sysupgade: How it works" here https://openwrt.org/docs/techref/sysupgrade, but my question seems hidden in step 19:

19. Write the upgraded firmware to disk. If platform_do_upgrade is defined, then ...

How is "Write the upgraded firmware to disk" performed?
By overwriting root & boot partition separately or by overwriting the whole disk?

If it's the whole disk, I fear that the sysupgrade process may be completely unsuitable for NAS devices where the partition table and a data partition must absolutely be preserved during upgrade.

Thanks!

2

Replying to my own post since I did some digging.

It seems as if a sysupgrade would wipe the whole disk, since the sysupgrade image contains a complete HDD image including boot sector and everything:

openwrt-23.05.0-rc2-apm821xx-sata-wd_mybooklive-squashfs-sysupgrade.img:
  DOS/MBR boot sector;
  partition 1 : ID=0x83, active, start-CHS (0x20,2,3), end-CHS (0x61,2,6), startsector 8192, 16384 sectors;
  partition 2 : ID=0x83,         start-CHS (0x82,0,9), end-CHS (0x3ef,2,62), startsector 32768, 2027520 sectors

However, when sysupgrading my own device, this is not what happens: After flashing the squashfs-sysupgrade image, my partitions are still there:

root@OpenWrt:~# gdisk -l /dev/sda
GPT fdisk (gdisk) version 1.0.9

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.
Disk /dev/sda: 15628053168 sectors, 7.3 TiB
Model: TOSHIBA HDWG480 
Sector size (logical/physical): 512/4096 bytes
Disk identifier (GUID): C1FB2C00-275D-49F0-957D-E5C88FF38B87
Partition table holds up to 128 entries
Main partition table begins at sector 2 and ends at sector 33
First usable sector is 34, last usable sector is 15628053134
Partitions will be aligned on 2048-sector boundaries
Total free space is 16350 sectors (8.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            8192           24575   8.0 MiB     8300  Linux filesystem
   2           32768         2060287   990.0 MiB   8300  Linux filesystem
   3         2060288         3108863   512.0 MiB   8200  Swap
   4         3108864     15628053134   7.3 TiB     8300  Data

Is this intentional (restoring the partition layout after a sysupgrade)?
This would be great! Or coincidence, possibly due to me switching the partition table to GPT layout?
Can I rely on a sysupgrade not destroying / removing additional partitions on the system block device? This would be mandatory for NAS devices like the MBL.

3

You question is too open...

For the majority of real routers with an embedded flash chip, there is no partition table on disk. The table for MTD is included in each firmware image. So, the generic sysupgrade always wipes it...

You are apparently talking about the minority of devices like x86, Nas, etc., where a real drive is on play. You might edit the title to be more specific to attract people knowing those devices types.

1 Like
4

I'd love to and have actually attempted to put my reply in the first post before replying to myself.

But I cannot edit it any more
The edit button has disappeared, I can only edit my reply.

Or am I missing something?

5

New users have restrictions on editing.
I edited the title slightly.

6

Hello:

Yet another happy OpenWRT user.
Because I was able to put in on my WD-MBL instead of having to throw in the bin.

OpenWRT adapted beautifully to what it may well be a small corner use case for it. ie: not a router, a limited hardware and unsupported NAS with no RAID capability.

But not so much of a minority: just how many thousands of these thinguies are still out there looking for a new lease in life?

... the majority of real routers ...
... devices like x86, Nas, etc., ... where a real drive is on play.
Indeed ...
The WD-MBL is certainly not a router and the OpenWRT firmware is written to HDD.

... edit the title to be more specific ...
I have just seen it this morning and don't know if it has been edited by the OP but it is clear to me.

That being the case, the question posed by the OP is one I also need an answer to.
Like most WD-MBL users out there running OpenWRT.

I installed OpenWRT on my WD-MBL and, after much mucking about/help with this forum, was able to modify a number of things (software/hardware) to solve some issues. eg: flexibility, graceful shutdown, bad shutdown recovery, etc.

The drive in the WD-MBL is the OEM 1Tb and I originally partitioned it in this manner:


After making a few changes and adding quite a bit of software from the repository, I ran out of space in the rootfs but solved the problem (off-line) with gparted.

Which brings me back to OP's question which specifically applies to this case.
ie: OpenWRT not flashed on a chip (like it would be on a router) but installed on a HDD (like it would be on an x86, Nas, etc. device).

My question is, does a sysupgrade wipe & reinstall

  • the entire disk including partition table (this would be bad) or
  • just the contents of the kernel and root partitions?

Can I rely on a sysupgrade not destroying / removing additional partitions on the system block device? This would be mandatory for NAS devices like the MBL.

While "people knowing those devices types" may have an answer based on their experience with a WD-MBL+OpenWRT, I think that only the devs who wrote the sysupgrade routine/scripts have the right/definitive answer to this question.

Thanks in advance.

Best,

PCL

1 Like
7

hi,
in my experience on x86 sysupgrade will "wipe" whole disk, meaning it writes the image file onto disk as is. if you upgrade from official image this means partition table and partitions alignment will be the same.

you can verify it easily, modify the installed partition layout, e.g. increase root partition (optionally you can increase filesystem too), then do sysupgrade. the partition will be reset to official size (~104MB).

this process has a benefit/side effect though: what you do beyond sda1+sda2 sysupgrade does not care.

but fire up owrt in virtualbox/vmware/whatever and you can test it.

8

Like I said above later, in my second message...

(as the OP apparently has no rights for that)

9

Hello:

Indeed ... 8^)

Seems I was writing/posting while you did that and did not see it.
Thank you for the heads up.

Best,

PCL

10

Hello:

Right ...
So the /dev/sda2 partition will go back to to it's original ~104Mb size albeit leaving /dev/sda3 untouched.

Yes?

That means that before doing anything else, I would have to resize it (104Mb -> 200Mb), lest I run out of space while adding software to the standard WD-MBL image.

That would mean taking the HDD out of the WD-MLB box and working in it off-line with gparted.

A way out of what is a bit of a hassle but sort of second nature if you have done it a few times, would be to resize the upgrade snapshot before using it.
ie: burn it to an SD card/USB stick, resize the rootfs and save it as an *img file.

That way, the image being written to disk would be the same size as the partition it was being written to and taking apart the WD-MBL would be avoided.

Next step would be the installation of all the stuff you added and the configuration files which is another can of worms.

I'd say it is all something to do if really necessary, like security fixes and justifiable kernel upgrades, rather doubtful for this hardware.

I'd appreciate your comments on this.

Thanks in advance.
Best,

PCL

1 Like
11

yes, sda3 will be untouched. and yes, sda2 will be the official size. so you need to extend it again if default 104MB is not enough.

or you create a custom firmware with larger root size and install that first, which will adjust partition table, then sysupgrade to your custom larger rootfs image.

or use the forum search as there are x86 upgrade topics :wink:

1 Like
12

I tried this yesterday evening on Ubuntu 22.04:

# 1. Install using customized image & packages
cd ~/tmp
wget https://downloads.openwrt.org/releases/23.05.0-rc2/targets/apm821xx/sata/openwrt-imagebuilder-23.05.0-rc2-apm821xx-sata.Linux-x86_64.tar.xz
tar xJf openwrt-imagebuilder-23.05.0-rc2-apm821xx-sata.Linux-x86_64.tar.xz
cd openwrt-imagebuilder-23.05.0-rc2-apm821xx-sata.Linux-x86_64

# 2. Set rootfs image size to 990MB
sed -i -e '/CONFIG_TARGET_ROOTFS_PARTSIZE=/s/=.*$/=990/;' .config

# 3. Create install image
PACKAGES=" ... "   # my custom packages list including samba4, rsyncd etc.
make image PROFILE="wd_mybooklive" PACKAGES="$PACKAGES" FILES="../files/" CONFIG_IPV6="n"

# 3. Write it to the harddisk temporarily connected to PC (sdc)
IMG=bin/targets/apm821xx/sata/openwrt-*-apm821xx-sata-mybooklive-squashfs-factory.img
DISK=/dev/sdc

sudo sgdisk -Z $DISK    # destroy existing partition tables, just in case
gunzip $IMG.gz
sudo dd if="$IMG" of=$DISK bs=10M
sgdisk --mbrtogpt $DISK
sudo sgdisk --new 3::+512M --typecode 3:8200 --new 4:: --typecode 4:8300 $DISK
sudo sgdisk --change-name=3:Swap --change-name=4:Data $DISK

mkswap ${DISK}3
mkfs.ext4 -O bigalloc -C 65535 -m 0 -L Data -E lazy_itable_init=0,lazy_journal_init=0 -v /dev/sda4

# 4. Remove HDD from PC, attach to MBL board, power on
ssh root@openwrt

This worked fine and the HDD partition layout is now in GPT format (the install image was in MBR format).

Then I used LuCI to upload the sysupgrade image (same version) and performed a sysupgrade. The sysupgrade image was also in MBR format.

But after rebooting, the HDD partitioning was still in GPT format. And sda3 and sda4 also still existed and were detected and mounted.

So the harddisk cannot have been completely wiped.

My question is only, if I can rely on this behaviour in the future?

13

million dollar question :slight_smile: only devs know.

14

Hello:

Best,

PCL