Distcc is a program that allows compiling jobs to be distributed across multiple computers via a network connection. I got this working, but had difficulty finding any reference material. The ones I did find would not work for cross-compiling and would only be helpful during the tools and toolchain compiling stages. To that end, I'm posting this guide in case anyone similarly struggles.
I'm compiling on a WSL Debian Platform, though the instructions should be easy to adapt to others.
Note: In what follows, I will use the term "client" to refer to the main machine you are compiling on (the one that you type the "make" commands in), and "servers" to refer to the machines that are aiding in the compiling process. This DOES work with ccache.
For WSL, Mirrored networking mode doesn't appear to work with NFS (which is necessary), so the first step is to forward the distcc port (3632) from windows to WSL. If you are running straight linux, then obviously you can skip this part:
1.) Open an admin-terminal (powershell) on all hosts and type "wsl.exe hostname -I" to get the ip address of the WSL instance, say its 172.27.55.186.
2.) Run:
netsh interface portproxy add v4tov4 listenaddress=0.0.0.0 connectaddress=172.27.55.186 listenport=3632 connectport=3632
(Note: I've found that this needs to be redone each time the computer is rebooted, even if the rule still shows as active and the ip address is the same. To delete this rule, run "netsh interface portproxy reset" and to see all rules run "netsh interface portproxy show all").
3.) I was not able to get NFS working on WSL without systemd. To enable it, add "systemd=true" to the [boot] section in wsl.conf. Then exit and run "wsl --shutdown" and then reopen wsl.
4.) Add the following entry (with appropriate substitutions) to /etc/exports on the client:
/home/david/openwrt *(ro,sync,insecure,subtree_check)
** The insecure option allows connections from ports above 1024 and I found to be necessary because of how WSL operates. If you are using straight linux you likely won't need it.
5.) Add the following entry to /etc/fstab (appropriately substituted) on all servers:
client_hostname_or_ip:/home/david/openwrt /home/david/openwrt nfs ro,sync,user,noauto 0 0
6.) Export the directory on the client and mount it on all servers. The path DOES need to be identical.
7.) Install distcc on all hosts. Open /etc/default/distcc and set the "STARTDISTCC" flag to true and set the ALLOWEDNETS variable (keep 127.0.0.1). For me, it was
ALLOWEDNETS="127.0.0.1 172.27.0.0/16 192.168.1.0/24""
8.) On the client,
export DISTCC_HOSTS="server_ip_addresses localhost"
9.) Start the distcc service on all hosts. Run the lsdistcc program on the client to make sure they can be reached.
10.) On the client, navigate to staging_dir/host/bin. You will see symlinks for gcc and g++. Note on mine gcc was actually symlinked to /usr/bin/cc and not /usr/bin/gcc. Remove those symlinks and recreate them to point to the corresponding program in /usr/lib/distcc.
e.g. rm g++ && ln -s /usr/lib/distcc/g++ g++
11.) On the client,
make -j(numjobs) tools/compile && make -j(numjobs) toolchain/compile
12.) Navigate to staging_dir/toolchain..../bin. You will see symlinks with names like "aarch64-openwrt-linux-gcc" pointing to files with names like "aarch64-openwrt-linux-musl-gcc" in the same directory. Create a file (mode 777) distcc-gcc:
#!/bin/sh
distcc /home/david/openwrt/staging_dir/toolchain.../bin/aarch64-openwrt-linux-musl-gcc "$@"
13.) Create similar files for c++,g++, and gcc-13.3.0 (or whatever version shows in there).
14.) For each of the four files, repoint the symlinks to their distcc counterparts:
e.g. rm aarch64-openwrt-linux-gcc && ln -s distcc-gcc aarch64-openwrt-linux-gcc
15.) make -j(numjobs) world
16.) To monitor the progress, run on the client (has to be same user that ran make command) "distccmon-text 1"