Building on FreeBSD

And just about every other GNU-ism out there (including bash, as I recall), not to mention that LLVM has completely replaced gcc.

My experience is like that of @diizzy -- either run a multi-core VM, dual boot, or spin up a cheap Debian box, either physical or cloud-based.

mkhash on freebsd

https://bugs.openwrt.org/index.php?do=details&task_id=971&order=category&sort=desc

1 Like

After installing the coreutils and various GNU versions of common Unix utils I get as far as this when running

./scripts/feeds update -a 
Create index file './feeds/packages.index' 
Checking 'working-make'... ok.
Checking 'case-sensitive-fs'... ok.
Checking 'proper-umask'... ok.
Checking 'gcc'... ok.
Checking 'working-gcc'... ok.
Checking 'g++'... ok.
Checking 'working-g++'... ok.
Checking 'ncurses'... ok.
Checking 'perl-thread-queue'... ok.
Checking 'tar'... ok.
Checking 'find'... ok.
Checking 'bash'... ok.
Checking 'patch'... ok.
Checking 'diff'... ok.
Checking 'cp'... ok.
Checking 'seq'... ok.
Checking 'awk'... ok.
Checking 'grep'... ok.
Checking 'getopt'... ok.
Checking 'stat'... ok.
Checking 'unzip'... ok.
Checking 'bzip2'... ok.
Checking 'wget'... ok.
Checking 'time'... failed.
Checking 'perl'... ok.
Checking 'python'... ok.
Checking 'git'... ok.
Checking 'file'... ok.
Prerequisite check failed. Use FORCE=1 to override.

Can't figure out why 'time' failed...

You need GNU time or busybox time to be used, not a shell internal or other substitute.

Most of the other GNU utils are available as FreeBSD pkgs, for example tar is available as gtar, but I couldn't find anything which would let me install GNU time. However the FreeBSD man page for time
https://www.freebsd.org/cgi/man.cgi?query=time&apropos=0&sektion=0&manpath=Red+Hat+Linux%2Fi386+8.0&format=html

mentions GNU time but I can't work out how to use it.

See the commit adding busybox time for reference, if your BSD time implementation supports the -f parameter and syntax, you could add an equivalent patch checking for it:

The requirement for a full time implementation was added in:

https://www.gnu.org/software/time/

The checking for a version of time that supports the -f option is very fragile. If you install another version, I'd suggest blowing away your staging_dir/ as (a) the implementation just checks that busybox time returns BusyBox and not that it was successful in using the -f flag, and (b) the prereq check is "sticky" and doesn't seem to adapt to changing environments too gracefully.

-f is, unfortunately, GNU-specific and the package for busybox on FreeBSD doesn't supply the time applet.

It looks as though there is a single invocation of env time in include/subdir.mk

Given that it appears that Linux time and FreeBSD time both support the POSIX-compliance flag of -p

       -p, --portability
              Use the following format string, for conformance with POSIX standard 1003.2:
                        real %e
                        user %U
                        sys %S

it seems as though a more portable approach would be to use that format and use a small script (or make magic) to transform it into giving the same results as (from include/subdir.mk)

env time -f "time: $(1)$(if $(4),/$(4))/$(if $(3),$(3)-)$(2)\#%U\#%S\#%e"

Given that we already require Perl and Bash in the Buildroot I wonder if the most sane path forward would be to either write the timing stuff in Perl or do a Bash script to do the command timing.

Requiring some unicorn program here almost nowhere present by default does not seem sane to me, it also caused quite some grief on the Buildbots back when it was introduced.

1 Like

According to the guide mentioned in the first post

The build system works on Linux, BSD or MacOSX operating system.

so I'm assuming that it did work as is at one and then it broke. Is it fair to draw that conclusion?

If so what changes were made to break it?

Building on FreeBSD is tested very infrequently. I think I spent some time to make the base build complete on FreeBSD a few years ago but I am quite sure that many issues crept back in the meanwhile.

As far as I am aware no single developer productively uses OpenWrt on FreeBSD (or any other non-Darwin BSD) so the port will get stale fast and newer versions of the various utilities tend to reintroduce GNUisms and other non-portable solutions which require constant patching.

Quite some amount of development happens on OS X though so the base build is supposed to work fine on Darwin at least. The wider package feed universe is another matter though, I don't think that anyone actually frrequently build tests all packages on Darwin.

Is there a changelog which shows the changes since it last worked on FreeBSD?

Is the project meant to be Posix compliant and does anyone ensure that it is?

Basically "git log" since the last round of freebsd fixes. The last thing I did was 2adab34c20 imagebuilder: use FreeBSD compatible tar invocation in February 2015.

No and no.

Before I dig into this further (as I agree with your assessment), a couple questions about how the build system is deployed and used, if you've got the time (no pun intended).

time: package/base-files/compile#0.14#0.04#0.81

Are the logs automatically parsed for this timing data? If so, is there reasonable flexibility in the format of the timing information?

I'm guessing the output was put in a single line for extraction of the timing data with a simple grep or sed kind of script. However it looks like there is one timing line in the individual compile.txt logs that seem to be output on a package-by-package basis, so it's not as though one is extracting from a monolithic log file and could "simply" take the last three lines of the file, if the POSIX format were used.

Is there anything I can try amending to see if I can get around the time problem on FreeBSD.

Not really sure where the

Checking 'time'... failed.

originates


You can install busybox using ports as (GNU)time isn't ported and BSD doesn't have the same output layout

busybox doesn't have a time applet that supports -f on both Debian and FreeBSD (and likely more). The above-mentioned pre-req check fails miserably everywhere, as busybox time returning BusyBox doesn't mean that busybox time -f <format string> is functional.

jeff@deb-devel:~$ busybox time
BusyBox v1.22.1 (Debian 1:1.22.0-19+b3) multi-call binary.

Usage: time [-v] PROG ARGS

Run PROG, display resource usage when it exits

	-v	Verbose

jeff@deb-devel:~$ busybox time -f '%E' echo fails            
time: invalid option -- 'f'
BusyBox v1.22.1 (Debian 1:1.22.0-19+b3) multi-call binary.

Usage: time [-v] PROG ARGS

Run PROG, display resource usage when it exits

	-v	Verbose

@balanga -- basically you need to remove the check for the GNU-specific, non-standard behavior of time -f from prereq-buid.mk and change the call in include/subdir.mk to remove the -f flag and its argument.

It is possible that there is code that parses all of the logging output files in the public repo. I haven't checked, but it should be easy enough to find if it bombs at the end of the run. As unless you've explicitly enabled file logging there won't be log files, it shouldn't be a problem even if there is such code.

1 Like

I implemented a Perl based drop-in replacement for that time command in master now.

3 Likes

What was the reason for the second thread on this issue: OpenWrt Build System on FreeBSD ?

1 Like

The first was to find out if anyone had it working, the second was basically an announcement that I had actually come up with something should anyone be interested in trying it out.