well.... we should check if the firmware recreate them too...
I'm fine with keeping the reserved partition, but I would like to point out that the NETGEAR firmware offers a possibility to back up the configuration:
Additionally, I find it a bit odd that the reserved partition doesn't cover all of the OEM partitions succeeding the netgear partition.
It's missing:
You are wrong.
"reserve" covers all those, and it is not missing anything.
cert, pot.bak, tracffic_meter +.bak and dongle are actually all part of the "netgear", like have I explained in the previous messages about "netgear".
mtd partition numbering does not correspond to the partition order on the flash. Note also that /proc/mtd does not show the start location, it only shows the size.
Kernel log shows the location better:
Openwrt:
[ 1.102678] 0x000000c80000-0x000001180000 : "APPSBL"
[ 1.111854] 0x000001180000-0x000001200000 : "APPSBLENV"
[ 1.113467] 0x000001200000-0x000001340000 : "art"
[ 1.118816] 0x000001340000-0x000001480000 : "artbak"
[ 1.123597] 0x000001480000-0x000001880000 : "kernel"
[ 1.133283] 0x000001880000-0x000003480000 : "ubi"
[ 1.181240] 0x000003480000-0x000007900000 : "netgear"
[ 1.297114] 0x000007900000-0x000008000000 : "reserve"
[ 1.309652] 0x000001480000-0x000003480000 : "firmware"
You need to look at the start offsets to understand the "reserve" contents properly.
Like I explained above:
the “reserve” contains 4 actual OEM partitions (defined as 128 kB blocks):
OEM:
{"crashdump", 0x03c8, 0x0004},
{"language", 0x03cc, 0x001c},
{"config", 0x03e8, 0x0009},
{"pot", 0x03f1, 0x0009},
Those are 128 kB blocks, not 64 kB, so the hex math goes this way:
starts at 0x3c8 of 128 kB blocks = 0x7900000
size is 0x0004 + 0x001c + 0x0009 + 0x0009 = 0x0032 of 128 kB block = 0x0640000
So, those 4 start at 0x7900000 and their combined size is 0x0640000, which leaves some free space at the flash end (from 0x7f40000 to 0x8000000).
"reserve" has been defined as starting 0x7900000 and size 0x0700000, so even the remaining empty space is included in the definition.
1 Like
Sorry, it seems I got a lot of things mixed up. Thanks for taking your time to explain it =)
which dsa commit should i pick?
this or #338 in this topic
1 Like
I think that I tracked that down: the sysupgrade process uses ubi-based write tool "ubiupdatevol" instead of the old good mtd. Kernel and rootfs are written separately using that tool into an existing mtd partition, and the tool reads the mtd partition info from the currently running firmware. (there is no practical way to check inside the image for the new partition structure)
Apparently that tool also sets the ubi volume size info, so even if the new mtd partition is larger, the ubi volume is locked into its original smaller size.
platform_do_upgrade --> nand_do_upgrade --> nand_upgrade_tar
https://github.com/openwrt/openwrt/blob/a367645f23d2ed93ea29c7237fa1b2d6c3ded7e4/package/base-files/files/lib/upgrade/nand.sh#L273
It was maybe easier with the old approach (e.g. in ar71xx), where kernel and rootfs were just concatenated into one unified image.
(But there is also the "ubirsvol" tool included in the firmware, which is supposed to resize ubi volumes on the fly. So, it might be possibly to resize the volume after flashing to cover to the whole new area. But I have not tried that, and it might be that it is impossible to run it for the active rootfs partition.)
Well with sysupgrade we switch to ramfs so all ubi partition are unloaded
Think it's not a problem...
How can I test it? Also we should be able to scan the ubi size of the packed firmware and check if that is greater than the installed kernel
Will be difficult during sysupgrade, as you need to uncompress the firmware, then read and parse the binary DTS blob. I don't see that as easily feasible in the sysupgrade script.
After sysupgrade, in a running system it might be possible more easily to scrape the mtd partition size from /proc/mtd and compare that to the ubi volume size of the ubi volume in that mtd.
Testing could be done by flashing a large partition firmware image from a router running traditional small partition, and then after the boot use command line ubi tools to first try expanding that rootfs ubi volume.
ubirsvol is installed by default?
so we don't have some tool to just analyze ubi structure without decompress it?
Seems to be.
Just start looking/testing the ubi commands you have in your router...
ubinfo gives you that info, visible in this thread a few messages upwards
root@OpenWrt:~# ubinfo /dev/ubi0_1
Volume ID: 1 (on ubi0)
Type: dynamic
Alignment: 1
Size: 698 LEBs (88629248 bytes, 84.5 MiB)
State: OK
Name: rootfs_data
Character device major/minor: 248:2
one question: does the dts gets replaced? can we read it some way?
One solution would be to flash the firmware and then at the next boot do some check and verify the ubi partition size and resize it if needed?
The dts is appended to the kernel image and gets replaced with it.
so tell me if i'm wrong...
situation: firmware with 20mb flash (with netgear space reserved)
we flash an image with different dts... at next boot, we will have dts with extended partition but same ubi partition as he doesn't got resized?
Then why not include a preiint script that extend the ubi (if needed) volume before the mount of rootfs ? @hnyman
The question would be, why bother?
The vendor firmware only reserves 2,125 MiB for the kernel partition, older LEDE versions only 2.0 MiB - kernel 4.14 currently needs ~2.3 MiB, this needs a migration via tftp, there is not way around it. Given these new investigations that the "netgear" partition gets recreated by flashing the OEM firmware via tftp, it makes sense to take the opportunity of unavoidable breakage to change this as well. This means users of the stable releases will only see a single break, users of the master branch might have to go through this twice - but that's the risk of living on the edge... And switching to DSA will be (almost) equally disruptive, that's why all three changes are supposed to go into master and the 18.06 branch before its release, to reduce the number of disruptions to the normal (stable-) user.
The problem with fancy automatisms is that sysupgrade functionality is provided by the old, currently running, firmware - that limits the features available for the sysupgrade (and you really want to keep it as generic as possible), and takign DSA as an example, striving for perfectionism (transparent migration provided by a script) pretty much kills the migration completely (as no one has provided such a script so far - and I don't think it's possible (in a reasonable way) to provide such a script that can reliably migrate custom switch setups).
we should at least create something to not lock user outside of the modem... reset the config and add the lan interface to lan bridge
An interesting driver showed up in linux-next: https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/commit/?h=next-20180604&id=a4333c3a6ba9ca9cff50a3c1d1bf193dc5489e1c
I don't use my R7800 currently so I will not be porting this.
I recompiled another build today, loaded it over TFTP, but still no wifi. I tried configuring it manually, but the radios won't associate. A 'dmesg | grep ath10k' showed this:
[ 18.454636] cfg80211: exports duplicate symbol __cfg80211_alloc_event_skb (owned by kernel)
[ 18.541247] cfg80211: exports duplicate symbol __cfg80211_alloc_event_skb (owned by kernel)
[ 18.613827] kmodloader: 5 modules could not be probed
[ 18.613959] kmodloader: dependency not loaded cfg80211
[ 18.617947] kmodloader: - ath - 1
[ 18.622978] kmodloader: dependency not loaded mac80211
[ 18.626458] kmodloader: dependency not loaded cfg80211
[ 18.631397] kmodloader: dependency not loaded ath
[ 18.636724] kmodloader: - ath10k_core - 3
[ 18.641229] kmodloader: dependency not loaded ath10k_core
[ 18.645280] kmodloader: - ath10k_pci - 1
[ 18.650608] kmodloader: - cfg80211 - 0
[ 18.654661] kmodloader: dependency not loaded cfg80211
[ 18.658157] kmodloader: - mac80211 - 1
Any suggestions? Also, if this is not the correct forum for these types of issues, please let me know where I should direct my questions. Thanks everyone.
Do you have information how to upgrade r7800 with Dumaos ? Do I need a modded firmware ?
you have a really bad build... recompile from the start pls