Building OpenWrt source and git branching

I have a question regarding building OpenWrt source in my Linux/Windows system. I was not happy with Windows Subsystem for Linux so I decided to dual boot my laptop with Windows 10 as my main OS and Linux Ubuntu 18.10 being used for OpenWrt development. Now hard disk being the constraint here, I have 500gb hard and only 60gb space was allocated to Linux.

I installed a minimum version of Ubuntu because I did not need any much applications in Linux but anyway now I have 42gb total allocated to /home and 4gb to swap and 14gb to / root. I am keeping my OpenWrt build source in /home of course and right now I have 2 different copies of the source representing latest snapshots and stable and they take around 26gb of space (more or less, because this is what df -h shows in /home) still leaving 14gb free.

I also decided to link the dl folder in the stable source (from snapshots) because that would save much time for downloading the same things twice and of course save space, at least some of it. Is there any more that I can do to save space on the drive? Maybe linking some folders together so they dont need to contain the same things twice.

I would also like to know if I can keep two different builds together, for instance, if I first checkout the master and then build it and after that I checkout v18.06.2 and then build it again so would that save me any time in building source again? I think it takes me around 3 or more hours to build the source from scratch with -j 3 option and my CPU is around 85% usage and RAM, 8gb, almost at full with buffer and cache being highest.

So any thoughts for this? Thank you.

# this is the output after building the source
ahmar@ahmar-Inspiron-3521:~/Desktop/build-system/openwrt$ free
              total        used        free      shared  buff/cache   available
Mem:        8046412     1387668      186968      242368     6471776     6110996
Swap:       3906556        5388     3901168

ahmar@ahmar-Inspiron-3521:~/Desktop/build-system/openwrt$ df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            3.9G     0  3.9G   0% /dev
tmpfs           786M  1.6M  785M   1% /run
/dev/sda6        14G  7.4G  5.7G  57% /
tmpfs           3.9G   41M  3.8G   2% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           3.9G     0  3.9G   0% /sys/fs/cgroup
/dev/loop0       36M   36M     0 100% /snap/gtk-common-themes/1198
/dev/loop1       15M   15M     0 100% /snap/gnome-logs/45
/dev/loop2      141M  141M     0 100% /snap/gnome-3-26-1604/82
/dev/loop4       13M   13M     0 100% /snap/gnome-characters/124
/dev/loop3       13M   13M     0 100% /snap/gnome-characters/139
/dev/loop5      141M  141M     0 100% /snap/gnome-3-26-1604/70
/dev/loop6      2.3M  2.3M     0 100% /snap/gnome-calculator/238
/dev/loop7       91M   91M     0 100% /snap/core/6405
/dev/loop8       88M   88M     0 100% /snap/core/5662
/dev/loop9      2.3M  2.3M     0 100% /snap/gnome-calculator/260
/dev/loop10      43M   43M     0 100% /snap/gtk-common-themes/701
/dev/sda1       296M   32M  265M  11% /boot/efi
/dev/sda8        42G   26G   14G  66% /home
tmpfs           786M   11M  776M   2% /run/user/1000

rdfind or fdupes can find and replace duplicate files with hard links. I don't know for sure what happens if one of them is updated later, but my guess is that the new file will be rewritten and de-duplicated.

For build time, I would install ccache and enable it in your builds (Advanced configuration options (for developers) > Use ccache). I can build everything from scratch including the tool chain) in about 16 minutes on an AMD Ryzen 5 2600X. Now I know that's not a low-end machine, but I wouldn't expect hours.

Using ccache will also speed "alternate" builds.

I am too many sorry if I don't remove build_dir completely when making more than a few commits of change across branches or back in time, even with make -j12 clean download world. A few commits, or moving forward tends to be OK, crossing between master and openwrt-18.06 dicey, moving from Linux 4.14 to Linux 4.19, deadly.

I find using git worktrees a good way to handle multiple targets or the like. Using a worktree means you only have one copy of the source on your drive, and git fetch openwrt or the like works from any tree, as does updating the head of your local version of OpenWrt's master or openwrt-18.06, for example.

That sounds like a good thing.

I dont remove anything at all, actually I am using two copies of the source for two different things. But yes if I only make a few changes like adding a patch or two then building the source again only takes a few minutes, maybe 15-20 or so. I am using a Intel Core i3 3227u processor.

Well if I look at my current two devices, they fall under the 4.9 category for stable and 4.14 for snapshots so I think it will be problematic if I switch builds within the same source.

Well I am not really sure what git worktrees are. I would like it better if I only had one local copy of the source and could switch builds easily without having to build much of the source code but I think that between kernel versions it would be better to clean it up and start again and in that case I think I should stick with the two copies of the code because that way the builds will be faster (only a few minutes) than from scratch.

Using userspace based deduplication tools would be a very bad idea, GNU make depends heavily on timestamps to determine if a file has been changed since the last build - using your proposed approach would break that, by either needlessly recompiling components and/ or failing to notice that other parts need to be rebuilt.

They're magic!

Only half kidding...

One .git directory with the repo in it, then multiple "slave" trees in different directories with the "only" imitation being that you can't have the same branch checked out in multiple places.

So, for example, something like this (not tested, but should be close) would clone the main OpenWrt repo, create an env/ git repo for your .config and files/

git clone https://git.openwrt.org/openwrt/openwrt.git openwrt
cd openwrt
git remote rename origin openwrt    # since I have multiple remotes
git checkout -b master
git init env
touch env/.config
mkdir env/files
ln -s env/.config .config
ln -s env/files files

You work with the "env" repo from the top level with git -C env <some command>, pushd into it, or there are scripts that manage it, which are long forgotten as I've gotten used to

make menuconfig
git -C env diff
git -C env add -u
git -C env commit

or the like.

Oh, the magic!

Let's say that you want to work on an Archer C7

cd path/to/openwrt
git worktree add ../openwrt-archer-c7 my-archer-c7    # add <path to worktree> <branch name>
pushd env
git worktree add ../../openwrt-archer-c7/env my-archer-c7
popd
pushd ../openwrt-archer-c7
ln -s env/.config .config
ln -s env/files files

Now you've got openwrt for master, or what have you, and can fetch and pull there without disturbing your work. openwrt-archer-c7 shares the storage and branches with the "main" repo, as does openwrt-archer-c7/env with the other config directories. Only one copy of the .git directory (200 MB on my machine), no matter how many worktrees! Only one master to fetch / pull, and all your worktrees get the same. Same branches in all worktrees, so you can diff and cherry-pick between them.

I name mine all ~/devel/openwrt-something so I can tab-complete all my OpenWrt worktrees from my "main" development directory.

2 Likes