Using external toolchain

For Developers
1

Hello,
How can I properly set up the external toolchain for LEDE? Whenever I set up the external toolchain, the compilation just breaks.
Here is what I did -

  1. took a backup of the toolchain to /opt/blahblah
  2. tell (make menuconfig) that I am using external toolchain, the path is given /opt/blahblah, the prefix is given properly.
  3. The folder (/opt/blahblah) contains /lib and /bin (with musl in /lib)
  4. start compilation (make distclean, cp -> .config, make -j5 V=99)
  5. breaks at the start of kernel compilation giving an error for u32, u16 etc (asm-generic.h ?)

using at-91 as the target.

Any suggestion will be helpful.
Thanks and regards,

2

You need to build images using precompiled packages or also compile packages?
you need to use x86 64bit Debian or Ubuntu. I tend to use minimal net install Debian on a Virtual Machine for this kind of tasks.

You can see my notes about if you like: https://gist.github.com/braian87b/4a9048dcc2e2fd244f7d44b8896b7ccc

3

I want to use precompiled toolchain and (if possible host and libc (musl) tools). I want the lede system to only use significant time in compiling the packages, the kernel and the rootfs. I am using ubuntu mate (lite) (x86_64).

For example, it takes a quite a bit time to build a firmware image after a make distclean (or after a git checkout of lede). From the log, I can see that it uses a lot of time to build toolchain, host (cc) binaries and the libraries required to cross compile the packages.

I am using the LEDE way of adding packages (and have some custom packages).

Thanks again.

4

Compilation take time, what hw specs are you using?, may be you should disclean just remove the final compiled files, that should preserve the compilation cache objects, if not, google for "caching compilation" it should bring something. you could also recompile just wanted packages and preserve the rest precompiled.
Sorry but I don't know more about to help you with this... good luck!

5

Thank you for your help.
I am using at-91.

distclean removes everything (including toolchain, cached objects, musl library). I do only selected package build before building the rootfs (very quick). But in Jenkins it takes a whole lot of time to build (because it has to build the tc, host tools and libraries to perform cc). I was trying to reduce the time.

Thanks anyway for trying to give some direction.
Regards,

6

Then you should modify your files to avoid distclean being run, or even better, edit the command to replace it with you own rm command to directories with final compiled packages and directories of custom packages.

7

@fadedreamz
i have the same problem! Did you solved it?

I have change the configuration to use the external toolchain of lede which i compiled before.
I also changed the config options
"Target name" mips-openwrt-linux
"Toolchain prefix" mips-openwrt-linux-
and the kernel gets cross compiled with my externel toolchain.
But the userland failed with the first package!
The dependency are not achieved! libc.so is not found!

Package libnl-tiny is missing dependencies for the following libraries
libc.so

did you run into the same problem?

8

Hi all,
this is something I've been trying to set up as well for a while now for Continuous Delivery purpose.
I.e. I both need a clean build and for it to be as fast as possible. Up to now, I've ended up having to cache my toolchains but it's not that clean as I'm copying them in and out of projects.

I'm basically hitting the same issue as @feckert.

My current way of trying to get the external toolchain is as follow:

  • Get an initial build going to build the toolchain (using CONFIG_MAKE_TOOLCHAIN=y)
  • get the bz2 file from the bin directory and deploy somewhere (let's say /opt/cross)
  • Get a second build going with the external toolchain with the following (values would change depending on your target):
    • CONFIG_EXTERNAL_TOOLCHAIN=y
    • CONFIG_TARGET_NAME="mipsel-openwrt-linux-musl"
    • CONFIG_TOOLCHAIN_PREFIX="mipsel-openwrt-linux-musl-"
    • CONFIG_TOOLCHAIN_ROOT="/opt/cross/toolchain/lede-toolchain-ramips-mt7620_gcc-5.4.0_musl-1.1.16.Linux-x86_64/toolchain-mipsel_24kc_gcc-5.4.0_musl-1.1.16/"

I then run into the "libc .so missing for libnl-tiny" (or dnsmasq or whichever comes first).

I've found references to this issue including from people who can apparently address it (even though some may no be very clean either), here they are:

  • https://github.com/openwrt/openwrt/issues/129 (closed even though the issue doesn't seem to be addressed and the workaround mentioned doesn't speak to me i.e. I don't understand it)
  • http://blog.csdn.net/princeofdream/article/details/74733973 (looks like a great little blog post even though I don't speak chinese (so I stick it in g**gle translate) - it points to very interesting settings like CONFIG_LIBC_ROOT_DIR and CONFIG_LIBC_FILE_SPEC - but once more I can't seem to be able to find a fix when playing with them)

This is where I'm at and I think the light may come from the CONFIG_LIBC* settings (same ones exist for other libraries and are likely to need the same treatment).
I hope that helps. I'll update this thread if I progress any further. :smile:

1 Like
9

could only post two links in that post, but there's another reference:

10

@feckert
No, I didn't solve it. However, you can reduce Jenkins time by bz2 ing the staging_dir/host dir (It takes a lot of time to build it). Keep in mind that you also need the stamp dir to fake that build of your tools.

I am still looking for a proper way to do this, lede is probably broken when it comes to using the external toolchain. (I also mailed their developer list, but no response so far).

11

@VincentR
This is what I was trying to solve in the beginning, I can give you some clue regarding this.
If you use the external toolchain then the lede does not build libc. And I suppose you have to provide the cross-compiled musl library.

Even if you have the folder there, it won't pick up because (I am guessing) lede used hardcoded path for it?
I was able to reduce time by modifying the host-build.mk and force no compilation for host tools.
But obviously, this is not the proper way.

12

@VincentR
I have found a commit on lede master https://github.com/lede-project/source/commit/b6a8b43dd2771d4d237256417218bed862545ef4 which fixed the issue upstream with libc
Maybe this should be backported to lede-17.01?
Everything compiles well until I reach net-snmp package, which complains about any perl stuff which are not found!

13

Has anybody solved the "Package libnl-tiny is missing dependencies for the following libraries:libc.so.0" problem?? I tried with every kind of solution I found in this post but nothing seems to work..