Tutorial: Build custom Netgear R7800 firmware for a larger flash size/root space

ScreenShots

Hey

Please note that I'm not a native English speaker, so... :slight_smile:

I just bought a netgear R7800, with a 128M ROM. But after I upgraded its firmware to latest official LEDE, I found that it has only <20M free space left.
That's not a good start and makes me, feel depressed. And I don't want use some unstable tech like Extroot/Rootfs though.

After a 3 days research, I made a very perfect custom lede. Here I can't wait to share the detailed process with you.

Features

1. You will get about 80MB free space after use this custom build.
2. Inside the firmeware, it's exactly the same as the official build.
3. That means you can install any packages from the official downloads.lede-project.org, without dependency issue. (usually, when you build your own firmware, you can't use many pkg from downloads.lede-project.org. see: https://forum.openwrt.org/t/kernel-mismatch-on-self-compiled-firmware-and-module/1265)
4. And, you can always keep updated with the latest official releases.

How to

Now let's make R7800 great again.

  1. Setup basic environment

DO NOT USE root user! use a normal user which has sudo privileges instead.

For debian/ubuntu:

sudo apt-get install subversion g++ zlib1g-dev build-essential git python rsync man-db
sudo apt-get install libncurses5-dev gawk gettext unzip file libssl-dev wget
  1. Get code and checkout the official release tag

You can get release tags from github or https://git.lede-project.org/source.git. Here I'll use v17.01.2, the current latest version.

git clone https://github.com/lede-project/source lede
cd lede
git fetch --all --tags --prune
git checkout tags/v17.01.2
  1. Install feeds

./scripts/feeds update -a
./scripts/feeds install -a
  1. Fix .config to make sure that you have the same fingerprint as official build

# Please change 17.01.2 in the url accordingly.
wget https://downloads.lede-project.org/releases/17.01.2/targets/ipq806x/generic/config.seed -O config.seed
rm -rf .config*
mv config.seed .config
  1. Fix DTS, free your ROM space.

This is the most important part, please make sure you understand what I said before take actions.

Edit the following file with your favorite editor:

target/linux/ipq806x/files/arch/arm/boot/dts/qcom-ipq8065-r7800.dts

Original section is something like this(you can search for 'netgear'):

        nand@1ac00000 {
        ...
            kernel@1480000 {
                label = "kernel";
                reg = <0x1480000 0x0200000>;
            };

            ubi@1680000 {
                label = "ubi";
                reg = <0x1680000 0x1E00000>;
            };

            netgear@3480000 {
                label = "netgear";
                reg = <0x3480000 0x4480000>;
                read-only;
            };

            reserve@7900000 {
                label = "reserve";
                reg = <0x7900000 0x0700000>;
                read-only;
            };

            firmware@1480000 {
                label = "firmware";
                reg = <0x1480000 0x2000000>;
            };
        ...

Here I'll explain this section and tell you how to change it.

For example:

reserve@7900000 {           // reserve is the labeel, 7900000 means begin position, in HEX.
    label = "reserve";      // reserve is the label.
    reg = <0x7900000 0x0700000>;       // reg = < begin position, length of this partition >
    read-only;      // extra flag, don't change.
};

Another thing keep in mind:
firmware partition should always start from begin of kernel, and end to end of ubi. (Do the math yourself if I didn't make it easy to understand)

So, for R7800:

1. DON'T TOUCH kernel and ubi. and I won't touch reserve neither.
2. Remove netgear section. it contains, actually, nothing useful, if you want to use lede.
3. extend ubi partition to the end of original netgear partition.

Here is what I used finally:

            kernel@1480000 {
                    label = "kernel";
                    reg = <0x1480000 0x0200000>;
            };

            ubi@1680000 {
                    label = "ubi";
                    reg = <0x1680000 0x6280000>;
            };

            reserve@7900000 {
                    label = "reserve";
                    reg = <0x7900000 0x0700000>;
                    read-only;
            };

            firmware@1480000 {
                    label = "firmware";
                    reg = <0x1480000 0x6480000>;
            };

  1. make and compile.

make defconfig
make   # and you can use -j for optimized concurrency compile.
  1. Upload firmware to R7800

Firmware is compiled in

bin/targets/ipq806x/generic/lede-17.01.2-ipq806x-R7800-squashfs-factory.img

ALWAYS USE -factory.img instead of sysupgrade one, because factory.img apply new DTS(partition tables) while sysupgrade won't.

And I doubt this is the reason why official image didn't extend the root partition by default -- they just use the same structure as original product, so lede users are able to install LEDE via webpage.

Hope someone from dev team could explain.

Since TFTP firmware upgrading is pertty standard for a R7800 user, I won't write it here, see:

https://kb.netgear.com/22688/How-to-upload-firmware-to-a-NETGEAR-router-using-TFTP

  1. Done

Yes you just installed your custom and official repositry compatiable firmware. happy hack.

  1. About backup and Restore

Actually I didn't see the necessary backing up the original partitions, from another thread: https://forum.openwrt.org/t/netgear-r7800-only-19mb-flash-available/

Because,

  1. when you want to flash you router to any other lede release, just put it into TFTP mode and upload new -factory.img firmware.
  2. when you want to go back to netgear firmware, just flash an standard official lede firmware first, and then flash the netgear firmware again.

And, Netgear always provides latest firmware on their websites.

Summary

Thanks for @hnyman and many friends in this forum.

I read almost all threads about this topic, no one wrote a complete steps for newbies, hope this one helps.

1 Like

What's a reason to use 'menuconfig' after 'defconfig' if you don't change anything?

Sounds wrong.
As far as I know, factory and sysupgrade have no difference in partition handling.
DTS is included quite similarly in the kernel image in both images.

Usually the only difference is the OEM ID header so that the OWN flash routine will accept the factory image...

...is to maintain the possibility to fully revert back to the OEM firmware. Contents of the unused "netgear" partition have been left intact.

Oh definitely you're right. I'll remove it now. no need for that.

That make sense, thanks Hnyman.
I have another question, if I flash firmware with a different DTS structure in sysupgrade format via sysupgrade(lede webui), it leads a infinite unsuccessful boot loop on R7800, just like a bricked router.

If flashing firmware via sysupgrade take cares of DTS, how this could happen?

As far as I tested :slight_smile: log time ago (@hnyman, do You remember my post Netgear R7800 - only 19MB Flash available? ?) upgrading 'standard' LEDE image with modified DOES NOT change partition layout, only 'factory' image does it.

Hmm.
Then the difference is somehow in the mtd flashing, as the images themselves contain the same compiled Device tree (DTS / DTB).

The difference is just in the final packaging of the image:
https://git.lede-project.org/?p=source.git;a=blob;f=target/linux/ipq806x/image/Makefile;hb=HEAD#l61
https://git.lede-project.org/?p=source.git;a=blob;f=target/linux/ipq806x/image/Makefile;hb=HEAD#l214

Looking at the DTS extract above in post #1 (or the live DTS extract below), the partition "netgear" has been marked "read-only". That is likely the reason, why sysupgrade does not work.

partitions visible via the DTS in a live system:

ls -R /proc/device-tree/soc/nand@1ac00000/cs0/partitions/

It is possible that the OEM flashing and TFTP flashing routine allow erasing additional partitions, while LEDE mtd keeps some of them "read-only" protected (including "netgear").

So, it might be that you need to either

  • flash the factory image (via OEM tools) to override theread-only, or
  • first flash otherwise unmodified image that has "netgear"s "read-only" status removed and then use that to flash the sysupgrade.

Sorry for the noise :frowning: The original advice to use the factory image may well be valid.

Well, it doesn't matter if You flash 'factory' image over OFW or LEDE, it always change partition layout while sysupgrade don't.
I've checked it twice before posting my tutorial (in Polish, in my mentioned thread there is Google Translate 'd link).

Just wondering - why is this not in the official lede build for the R7800?

Discussed last time six mesages ago in this thread...

1 Like

You were too fast. I was about to delete my posting. Many thanks to you, shame on me.

Looking for some solution to get full flash without recompiling I put the script that converts ready made images:

Can someone share with me backup of the original "netgear" partion?

dd if=/dev/mtd7 of=/mnt/sda1/mtd7.backup

I have builded FW with changes from the first post. Something wrong with this patch. Lagging... Luci not working out from the box (see screen). Im receiving opkg lock error during package installation from GUI.

If i roll back changes - FW working as it shoud.
Can someone help me what i did wrong?

0091

root@LEDE:~# cat /proc/mtd
dev:    size   erasesize  name
mtd0: 00c80000 00020000 "qcadata"
mtd1: 00500000 00020000 "APPSBL"
mtd2: 00080000 00020000 "APPSBLENV"
mtd3: 00140000 00020000 "art"
mtd4: 00140000 00020000 "artbak"
mtd5: 00200000 00020000 "kernel"
mtd6: 06280000 00020000 "ubi"
mtd7: 00700000 00020000 "reserve"
mtd8: 06480000 00020000 "firmware"

root@LEDE:~# nandtest -k /dev/mtd8
ECC corrections: 0
ECC failures   : 0
Bad blocks     : 0
BBT blocks     : 0
06460000: checking...of 1)...
Finished pass 1 successfully

I try this for 18.06.0-rc2 but the partitions to change look differently. I have no idea what to delete / change:


partitions {
                                        compatible = "fixed-partitions";
                                        #address-cells = <1>;
                                        #size-cells = <1>;

                                        qcadata@0 {
                                                label = "qcadata";
                                                reg = <0x0000000 0x0c80000>;
                                                read-only;
                                        };

                                        APPSBL@c80000 {
                                                label = "APPSBL";
                                                reg = <0x0c80000 0x0500000>;
                                                read-only;
                                        };

                                        APPSBLENV@1180000 {
                                                label = "APPSBLENV";
                                                reg = <0x1180000 0x0080000>;
                                                read-only;
                                        };

                                        art: art@1200000 {
                                                label = "art";
                                                reg = <0x1200000 0x0140000>;
                                                read-only;
                                        };

                                        artbak: art@1340000 {
                                                label = "artbak";
                                                reg = <0x1340000 0x0140000>;
                                                read-only;
                                        };
                                         kernel@1480000 {
                                                label = "kernel";
                                                reg = <0x1480000 0x0400000>;
                                        };

                                        ubi@1880000 {
                                                label = "ubi";
                                                reg = <0x1880000 0x6080000>;
                                        };

                                        reserve@7900000 {
                                                label = "reserve";
                                                reg = <0x7900000 0x0700000>;
                                                read-only;
                                        };
                                };
                        };
                };

--> too
Is there a chance to change it in 18.06?

openwrt-18.06 and master give you 96.5 MB to play with on the r7800 by default, isn't that enough?

Yes, indeed - I have realised that I have 60 MB free after installing all required stuff.
For my understanding: Why is there only <100 MB reserved if I have a total of 500? Why is there so much needed for e.g. /tmp ?

500? R7800 has indeed 512MB but RAM :wink: , flash is "only" 128MB.

1 Like

Will we ever see this ported to the little sister device? The R7500v2. I don't think the partitions are different at all.