Building Packages on Apple M1 CPU

When build the tools, having issues detecting the proper bit size:

checking whether to install <libelf.h>, <nlist.h> and <gelf.h>... yes
checking for working const... yes
checking for off_t... yes
checking for size_t... yes
checking size of short... 0
checking size of int... 0
checking size of long... 0
checking size of long long... 0
checking size of __int64... 0
checking for 64-bit integer... no
checking for 32-bit integer... no
configure: error: neither int nor long is 32-bit
make[2]: *** [/Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13/.configured] Error 1
make[2]: Leaving directory `/Volumes/OpenWrt/openwrt/tools/libelf'
time: tools/libelf/compile#0.36#0.35#0.97
make[1]: *** [tools/libelf/compile] Error 2
make[1]: Leaving directory `/Volumes/OpenWrt/openwrt'
make: *** [tools/install] Error 2
3 Likes

Oh, is this natively or using docker for mac/hyperkit ?

This natively on my machine. Not using a docker environment or virtual machine.

I think OpenWRT devs might be overstating how "supported" other os-es are in the build instructions. Afaik continuous integration bots run the building in docker images based on debian:10 there's no integration testing involving any other os or platform other than amd64

See:

https://git.openwrt.org/?p=buildbot.git;a=blob_plain;f=docker/buildworker/Dockerfile

Interestingly, they're not using a snapshot of the Debian repos


They also provide: https://hub.docker.com/r/openwrtorg/sdk , not sure if it's the same

I was trying to avoid the docker environment. But I will attempt it now. I would love to get it built on macOS M1.

Considering the M1 arch just dropped, this is a little disingenuous.

checking size of short... 0
checking size of int... 0
checking size of long... 0
checking size of long long... 0
checking size of __int64... 0
checking for 64-bit integer... no
checking for 32-bit integer... no
configure: error: neither int nor long is 32-bit

These header defines are empty. This seems like a gcc issue. It could be that the tools are simply not available for the M1 chip yet, or is missing the proper pre-reqs for the system.

I've built OpenWrt on stand-alone Ubuntu, stand-alone Alpine, and Ubuntu & Alpine under under WSL2. But, all are x86/amd64 because up until now, a MIPS/ARM based system just wasn't worth the hassle to use as a BUILD system..

Anyone want to buy/donate a M1 chip Mac to the project? I'm sure someone would be willing to get it working :smiley:

1 Like

LOL - such a bad idea.

People will get M1 MacOS devices over time and eventually someone will get everything working with recent enough GCC.
If you're eager to work on this, but don't want to buy one or can't fork out 800eur for a cheapest Mac mini, you can rent one for 10c/hour from scaleway or macstadium or similar.

...and if you're serious, you might just setup some kind of test runner to cover MacOS building in order to ensure it works, so the openwrt project can just pay the bill for Mac mini time once in a while, and not have to deal with owning hardware.

Oh, I am absolutely not interested in developing on a Mac of any type, Intel or M1 :slight_smile:

The point was that the ARM M1 chip is fantastically new, and that error seems like a GCC error. While I would never expect someone to ACTUALLY ship an M1 Mac to someone, I'm also sure I've seen less crazy ideas actually happen. Building on ARM is going to be different than x86/amd64, and I honestly thing I just pinged on the below more than anything, and the easiest way to alleviate the below situation is for someone to proffer up the cash if they don't want to provide the knowledge or skill (which happens far more than I like to think about) :slight_smile:

Oh, I realize I may have come off as complaining that openwrt isn't supported on more host platforms. That wasn't really my intent - it'd be a lot of work for not that much gain from what I understand.

I was complaining more about the documentation.

Specifically how there's all kinds of dubious claims that something is "supported" in the wiki, but in a majority of cases it really means "it worked once for someone in the past".


TBH, I think focusing on Linux as /the/ supported host os for builds makes sense.
But I also I think tying it down to amd64 (or x86_64) and not at least investigating 64bit arm in 2021 is slightly weird.

(nowadays there's all kinds of Amazon gravitons, gigabyte ampere altra, and M1, and smaller SBCs like 8GB Raspberry Pi or various amlogic s922 and rockchip rk3399 based boards that could build a typical openwrt image in 20min to an hour off of a usb3 hdd).

Ensuring the tool chain perpetually buildable using various versions of macos would be a lot more work, than just ensuring everything works in some kind of chroot/vm like environment at 99.9% native performance.

I think ensuring it can work on arm64 / aarch64 in docker, or in a VM is probably less work and where I'd pour the effort if it was up to me. (..or if I had any energy/effort left to put into this).

Who said that it would be?

It's just not tested - and therefore you're more likely to be the first to encounter bugs, but it should work. Case in point, for a (very short) while a PPC64LE buildbot was used to build OpenWrt, which worked - but 'failed' when producing the imagebuilder binaries (well, it successfully built them, but obviously for PPC64LE and not x86_64, which made them rather useless for the public at large). Proving if it actually does, or what the issues are is up to those who actually want to use that hardware.

Why would it be weird?

At least so far, there was not a single ARM device available to mere mortals on a budget that could compete in terms of build (CPU, I/O, RAM) performance with even an aged mid-level x86_64 system. The mythological ARM servers which have been promised to conquer the market $next_year for at least half a dozen years still don't exist (unless you're a huge company with deep pockets to drive development yourself). While the RPi4 or Rockchip RK3399 are certainly a step into the right direction, they still can't really compete with an entry level ryzen system, even more so once you look at RAM sizes and I/O options (SATA, PCIe, …).

In terms of availability and performance, the Apple M1 could be a game changer, but it has just entered the market half a year ago - and prices and lacking Linux support don't really make this a straight forward choice at this point. I'm sure someone will try building OpenWrt on their shiny new M1 one day, just don't expect anyone to spend their own money to do it for you (while it's still new and expensive).

The same goes for renting CPU cycles on ARM based data centres, it costs money (the project doesn't really have an endless supply of) for rather little reward, it's not cheaper than renting x86_64 based resources, nor any faster, nor providing any other tangible benefit at this moment. The official binaries would still have to be built on x86_64 in order to provide imagebuilder binaries that are actually useful to the enduser - so trying it elsewhere would be a rather pointless exercise of proving that it works (which it should in general, you're just more likely to encounter issues <-- patches welcome).

If ARM is here to stay on the (consumer/ prosumer/ developer desktop and workstation) market, it needs to become competitive. What does this mean? Well, you'd need to be able to buy such a device that's roughly on par (performance, RAM, I/O, storage) with contemporary x86_64 systems, for comparable prices (+/- 15%-20% at most) - and it must also scale with the x86_64 offerings (from entry level budget to mid- to highend consumer devices; let's say i3, i7, ryzen competitors). And something like SBSA compliant devices must finally become available to mere mortals on a private budget, meaning that you can install any of your preferred linux distributions without having to think twice.

Apple with the M1 is certainly one alternative here, but at this point not really the natural choice for anyone wanting to do develop linux systems - you'd have to pay the early adopter's tax twice, once at the sales counter, another time trying to hunt down all the little warts and kinks before you get the build environment working.

RPi4 and RK3399 (etc.) have certainly paved the way to consider looking into ARM, but their performance is far from sufficient to do anything but one-off proof of concept builds. Sure, you can do that, if that's what you want to prove - or you could do actual development on OpenWrt.

Time will tell if ARM can become competitor (or of nVidia smothers the competition), once it does, it will find the way onto developers' desks quite naturally.

--
Disclaimer: I'm not an OpenWrt developer and can't speak for the project. But to the best of my knowledge, none of the OpenWrt developers are currently in possession of Apple M1 hardware.

The reason I am typing up my development skills by cross-compiling openwrt on M1. I am coming from C# Windows environment. This is main reason why I am using a M1. So I will take any suggestions to try. And hopefully I could submit a patch. :grinning:

Is all the checking lines coming from autoconf?

I think it's worth investigating:

Example 1: typical rack server

https://happyware.com/uk-en/r272-p30-gigabyte-arm-server-ampere-altra-q80-33

Example 2:
https://aws.amazon.com/ec2/spot/pricing/ (wish there was a table with CPUs/ram and pricing alongside) in case your workloads don't need to be available to you immediately - build farm.

Example 3:
Mac mini - it's $1500 for 16G/10GigE/M1 chip... ; not comparable to a threadripper in sheer brute force, but then again threadripper build usually doesn't go under 100W and can easily suck up 250-300W, depending on where you live this 300W of CI build workload can cost you $600/year (factoring in AC), and if you drop the total system cost of an x86 machine down to a $1000 you end up with an older 8 core.

With rumors of apple filling the gap in this 1000-5000 market soon with next generation of their chips, and rumors of northern hemisphere developers going back to hacking while sipping lattes with both themselves and their laptops sweating in the sun later this year... it might be worth investigating how it'd look like if openwrt would get on the bandwagon.

Yes, it's as if your homebrew environment is missing stuff that should be there in the standard library. Is cc in your case clang or gcc?

This problem it is not only on macOS with Apple M1 CPU. I have MacBook pro 2019 and when i compilated openwrt on TP-LINK 1043 on my computer i recevied this fail

Making all in tests
make[6]: Nothing to be done for `all'.
Making all in doc
make[6]: Nothing to be done for `all'.
Making all in tools
Making all in bench
make[7]: Nothing to be done for `all'.
make[7]: Nothing to be done for `all-am'.
make[6]: Nothing to be done for `all-am'.
touch /Volumes/OpenWrt/openwrt/build_dir/host/mpc-1.1.0/.built
CFLAGS="-O2 -I/Volumes/OpenWrt/openwrt/staging_dir/host/include " CPPFLAGS="-I/Volumes/OpenWrt/openwrt/staging_dir/host/include " CXXFLAGS="" LDFLAGS="-L/Volumes/OpenWrt/openwrt/staging_dir/host/lib " /Library/Developer/CommandLineTools/usr/bin/make  -C /Volumes/OpenWrt/openwrt/build_dir/host/mpc-1.1.0  install
Making install in src
/usr/local/bin/gmkdir -p '/Volumes/OpenWrt/openwrt/staging_dir/host/lib'
/usr/bin/env bash ../libtool   --mode=install /usr/local/bin/ginstall -c   libmpc.la '/Volumes/OpenWrt/openwrt/staging_dir/host/lib'
libtool: install: /usr/local/bin/ginstall -c .libs/libmpc.lai /Volumes/OpenWrt/openwrt/staging_dir/host/lib/libmpc.la
libtool: install: /usr/local/bin/ginstall -c .libs/libmpc.a /Volumes/OpenWrt/openwrt/staging_dir/host/lib/libmpc.a
libtool: install: chmod 644 /Volumes/OpenWrt/openwrt/staging_dir/host/lib/libmpc.a
libtool: install: ranlib /Volumes/OpenWrt/openwrt/staging_dir/host/lib/libmpc.a
make[6]: Nothing to be done for `install-data-am'.
Making install in tests
make[6]: Nothing to be done for `install-exec-am'.
make[6]: Nothing to be done for `install-data-am'.
Making install in doc
make[6]: Nothing to be done for `install-exec-am'.
/usr/local/bin/gmkdir -p '/Volumes/OpenWrt/openwrt/staging_dir/host/share/info'
/usr/local/bin/ginstall -c -m 644 ./mpc.info '/Volumes/OpenWrt/openwrt/staging_dir/host/share/info'
install-info --info-dir='/Volumes/OpenWrt/openwrt/staging_dir/host/share/info' '/Volumes/OpenWrt/openwrt/staging_dir/host/share/info/mpc.info'
Making install in tools
Making install in bench
make[7]: Nothing to be done for `install-exec-am'.
make[7]: Nothing to be done for `install-data-am'.
make[7]: Nothing to be done for `install-exec-am'.
make[7]: Nothing to be done for `install-data-am'.
make[6]: Nothing to be done for `install-exec-am'.
/usr/local/bin/gmkdir -p '/Volumes/OpenWrt/openwrt/staging_dir/host/include'
/usr/local/bin/ginstall -c -m 644 src/mpc.h '/Volumes/OpenWrt/openwrt/staging_dir/host/include'
mkdir -p /Volumes/OpenWrt/openwrt/staging_dir/host/stamp
touch /Volumes/OpenWrt/openwrt/build_dir/host/mpc-1.1.0/.built
touch /Volumes/OpenWrt/openwrt/staging_dir/host/stamp/.mpc_installed
make[3]: Leaving directory `/Volumes/OpenWrt/openwrt/tools/mpc'
time: tools/mpc/compile#13.00#7.11#21.84
make[3]: Entering directory `/Volumes/OpenWrt/openwrt/tools/libelf'
. /Volumes/OpenWrt/openwrt/include/shell.sh; gzip -dc /Volumes/OpenWrt/openwrt/dl/libelf-0.8.13.tar.gz | tar -C /Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13/.. -xf -
[ ! -d ./src/ ] || cp -fpR ./src/* /Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13

Applying ./patches/900-fix-undefined-macro-access.patch using plaintext:
patching file lib/elf_repl.h
patching file lib/gelf.h
patching file lib/libelf.h
patching file lib/sys_elf.h.in
touch /Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13/.preparedc7cb973335f50d059afec009811c3578_6664517399ebbbc92a37c5bb081b5c53
(cd /Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13/; bash ./configure --target=x86_64-apple-darwin20.2.0 --host=x86_64-apple-darwin20.2.0 --build=x86_64-apple-darwin20.2.0 --program-prefix="" --program-suffix="" --prefix=/Volumes/OpenWrt/openwrt/staging_dir/host --exec-prefix=/Volumes/OpenWrt/openwrt/staging_dir/host --sysconfdir=/Volumes/OpenWrt/openwrt/staging_dir/host/etc --localstatedir=/Volumes/OpenWrt/openwrt/staging_dir/host/var --sbindir=/Volumes/OpenWrt/openwrt/staging_dir/host/bin --disable-shared --enable-elf64; )
loading site script /Volumes/OpenWrt/openwrt/include/site/darwin
creating cache ./config.cache
checking whether make sets ${MAKE}... yes
checking for gcc... gcc
checking whether the C compiler (gcc  ) works... yes
checking whether the C compiler (gcc  ) is a cross-compiler... no
checking whether we are using GNU C... yes
checking whether gcc accepts -g... yes
checking how to run the C preprocessor... gcc -E
checking for a BSD compatible install... /usr/local/bin/ginstall -c
checking for ranlib... ranlib
checking whether ln -s works... yes
checking for ANSI C header files... no
checking for unistd.h... yes
checking for stdint.h... yes
checking for fcntl.h... yes
checking for elf.h... no
checking for sys/elf.h... no
checking for link.h... no
checking for sys/link.h... no
checking if gcc can compile elf.h... no
checking for ar.h... yes
checking for libelf.h... no
checking for nlist.h... yes
checking for gelf.h... no
checking whether to install <libelf.h>, <nlist.h> and <gelf.h>... yes
checking for working const... yes
checking for off_t... yes
checking for size_t... yes
checking size of short... 0
checking size of int... 0
checking size of long... 0
checking size of long long... 0
checking size of __int64... 0
checking for 64-bit integer... no
checking for 32-bit integer... no
configure: error: neither int nor long is 32-bit
make[3]: *** [/Volumes/OpenWrt/openwrt/build_dir/host/libelf-0.8.13/.configured] Error 1
make[3]: Leaving directory `/Volumes/OpenWrt/openwrt/tools/libelf'
time: tools/libelf/compile#0.82#0.91#2.00
make[2]: *** [tools/libelf/compile] Error 2
make[2]: Leaving directory `/Volumes/OpenWrt/openwrt'
make[1]: *** [/Volumes/OpenWrt/openwrt/staging_dir/target-mips_24kc_musl/stamp/.tools_compile_yynyynnyyyynyyyyynyynnyyyyyyyyyyyyyyyyyyyynyynynyyyynnyyy] Error 2
make[1]: Leaving directory `/Volumes/OpenWrt/openwrt'
make: *** [world] Error 2