Compiling OpenWrt Community build in the cloud?

My Linux "workstation" is an old laptop running 32-bit Debian on a 2-core Intel Atom processor. I've made four attempts to compile @Kong 's extensive and comprehensive version of OpenWrt for OnHub routers with NSS support each of which has taken almost two full days. I'm probably going to try a fifth time and I need to learn how to create or locate the log file that details what is going wrong.

But to save time and speed things up, I'm wondering if there is a service where I can connect to a high powered Linux machine in the cloud and do the compiling on a remote computer and just get the image binary downloaded locally. Is that a thing? Are there free services or is that a pay as you go, metered use kind of thing? What are the services - if there are some - and would it work?

Similar sized SSD drive upgrade likely would be most cost effective solution.

I don't think that would speed things up. My laptop has the space and has an SSD drive. Could the problem just be I need even more space on the drive? I might try to do it on a Raspberry Pi4, but I'll probably need a large microSD card. I normally just use a 32 Gb microSD. It's easy enough to flash a fresh Rasbian (Debian) OS without any GUI. The RP4 device has 4 CPUs with one thread each. My laptop has 2 CPUs with one thread each.

Compiling is I/O bound, then CPU, tmp.mount may help with I/O but your setup is missing in CPU head-room.
cloud depends on where you are, you can get decent usable linux vm for 1-10 bucks a month, cloud as you go will be same for day or week. (buck is average of dollar and euro)

This is literally what a cloud provider does (Linode, Digital Ocean, AWS, Azure, etc). They can hook you up to a proper Linux machine you can SSH into and do everything you would normally do on your laptop.

"Free tier" compute options are not a thing anymore, thanks to cryptocurrency miners. Cloud compute is by-the-hour, so just don't forget to fully turn off the VM after you're done. You can change the CPU allocation at any time, so start off with the smallest instance and scale up from there.

3 Likes

Haswell to kaby-lake entry-level i5 SFF systems from one of the big vendors with 8 GB RAM (often sold without disk, so add 15-20 bucks for a 256 GB SSD and maybe (ideally) another 18-25 EUR for a second 8 GB RAM module) are selling for around 30-50 EUR used, which is rather good value for pretty decent systems for running linux. These systems are very reasonable desktops (for basically all office'ish and media consumption uses; they are not certified for Windows 11, which makes them cheaper), but also still make very reasonable buildds/ buildhosts for OpenWrt and more; expect build times for a 'normal' OpenWrt image somewhere around 45 minutes +/-.

Your Atom N270 (and Atom 330 would not be much better) is just from another world in comparison and extremely slow.

Don't get me wrong, the sky is the limit and you can see quite big improvements with more modern gear (be it i7 within those old generations or even more with contemporary platforms), but medium performance systems since sandy-bridge are still coping quite well on linux nevertheless (even for infrequent building from source).

4 Likes

I agree with @slh on this, get yourself a computer made in the last 10 years and it will be a far less frustrating experience. A Ryzen 7 5700G, released in 2021 and nowhere near AMD's top tier of processors at the time, can compile all of OpenWrt in a several minutes.

To aid in your search, look for "tiny mini micro PC" (coined by ServeTheHome), especially on the usual places where you buy used stuff. These would fall under the "about 50 USD" category of PCs @slh was alluding to.

1 Like

tiny/ micro PCs would be USFF and usually come with a 50% markup compared to SFF systems (and often lower CPU performance). If you need their smaller size or their lower idle power consumption for 24/7 usage (e.g. to be used as router), they are interesting - but SFF systems tend to provide more performance for less money (and it's more 'normal' PC hardware, compared to the notebook style components for USFF systems, which makes it easier to extend/ upgrade/ repair than USFF systems).

SFF ~= half the size of a mini-tower; around mini-ITX size with slim/ half-height PCIe brackets
USFF ~= small cigar box

1 Like

My boneyard of computer junk is already large. The old laptop is when it's not being used for Linux stuff has been very reliable as a PiVPN server. There are some things that can ONLY be done on Linux and that old laptop has been very useful even at advanced age.

But, I'm looking at one of these that seem cheap and powerful:

Beelink SER5 Mini PC

However, I'm also thinking about getting one of those boxes that can run OPNSense or pfSense to create a powerful Firewall and Router and I could SSH into that to compile stuff when needed?

Looking at this one for that purpose:

P1 Mini PC 6W Low Power Intel N3050/N3160 Quad Core Quad Thread X86 Soft Router

Atom or Celeron are not for compiling.

2 Likes

Just to add to what brada4 rightly raised, these are some rough speed comparisons for compiling a distro-like linux kernel (v6.4) on various hardware. Take this only as a rough rule of thumb, as disks (well, all but the Atom 330 were running from SSD) and RAM sizes differ (and RAM has a larger impact on build times than you might think at first):

  • Atom 330 (Diamondville)
    real 674m32,640s
    user 2407m8,127s
    sys 243m21,977s
  • Celeron J1900 (Atom, Baytrail-D)
    real 213m36,622s
    user 639m13,289s
    sys 62m58,065s
  • i5-4440 (Haswell)
    real 49m54,129s
    user 176m2,935s
    sys 19m35,869s
  • i5-6400 (Skylake)
    real 50m5,662s
    user 173m51,155s
    sys 22m41,489s
  • i5-7400 (Kabylake)
    real 47m44,820s
    user 161m15,626s
    sys 19m43,846s

Consider that the Atom 330 has twice the number of cores than your N270 and is x86_64 capable (which improves performance as well),, so it should be more than twice as fast as your N270. The Atom 330 system above is mini-ITX based, so comes with adequate cooling (which I wouldn't expect for an N270 netbook either), I do not expect thermal throttling to hit in those figures.

I would rate the N3050/ N3160 to be roughly comparable to the j1900 in my list, yes it's a newer generation, but it's also the 'tablet' CPU compared to the desktop CPU (and as you see, performance has remained roughly consistent betwee haswell and kabylake, so there aren't major jumps to be expected for braswell relative to baytrail either.

Yes, this is not a benchmark for cross-compiling OpenWrt on x86_64 to ARMv7/ ipq806x, but it can provide some rough performance comparison.

2 Likes

Consider getting an old Intel Mac Pro for the job. They're actually quite fast when running Linux. Action Retro just bought a 2009 Mac Pro 4,1 for $100 USD to run Linux and said that it is an "absolute bargain considering the specs" (be sure to also budget a bit for an SSD if it doesn't already have one).

Fun fact, this is the same machine that I use today as my daily driver... it's really old and I need to replace it, but I've compiled previous versions of OpenWrt within an Ubuntu VM (via VirtualBox running on a Mac OS host) in well under 12 hours (edit: probably low single digit hours). Would have been much faster if I had been running linux natively.

moar cores

  • Ryzen 7950x, make clean, time make -j32 world
    real 5m4.180s
    user 61m1.101s
    sys 14m7.347s

If I do a make dirclean and rebuild the toolchain it takes about 15m.

2 Likes

I tend to use vps for compilation.
Over on https://lowendtalk.com there's lots of options, you can rent for 30 days etc, VPS with 8gb memory and enough SSD storage for compiling are around £8 for a month

1 Like