I was wondering if with the compiled kernel of lede is possible to run the docker engine and spin up containers.
Do you know about any initiative about this?
I'm not aware of anyone working on it, but it shouldn't be too difficult. OpenWRT, being based on Linux + Musl + Busybox, has much in common with Alpine Linux. Alpine is very popular as a container base and can also also be used as a host:
http://web.ist.utl.pt/joao.leao.guerreiro/post/alpinedocker/
Clearly this means Docker Engine compiles OK against Musl libc, so it should be possible to build it on OpenWRT without problems.
I don't know whether OpenWRT has the necessary features enabled in the standard Kernel configuration, but it's easy to build OpenWRT from source, and the build system allows full access to Kernel options via "make menuconfig".
Some useful info about Docker Kernel config here:
https://wiki.gentoo.org/wiki/Docker#Kernel
One thing to be aware of is that Docker Engine itself is pretty hefty. The statically compiled binaries here should give you an idea of the size:
https://download.docker.com/linux/static/stable/x86_64/
Compiling against Musl may reduce the size, and perhaps there are compile options that reduce it further, but still it is rather impractically large for the limited hardware OpenWRT is targeted at. This, along with the large size of typical containers, results in very limited use cases for Docker on OpenWRT, which is probably why no-one has bothered yet. Still, if the hardware you're running OpenWRT on has sufficient capacity, it should work.
1 Like
We had to enable some kernel configuration options and for some reasons I can't remember it required glibc instead of musl. We have run docker images on ARM platform with glibc running on latest openWrt.
This is a good starting point
3 Likes
I note that this was 3 years ago. It would appear that whatever issues there were with Musl have been fixed, because Alpine are definitely building it against Musl:
http://build.alpinelinux.org/buildlogs/build-edge-x86_64/community/docker/docker-18.02.0-r0.log
Still, the result is a horrendous 42.91 MB package that takes up 147.61 MB uncompressed 
This is clearly down to docker being written in Go, which is notorious for producing massive binaries.
Any news around here? Since we got Golang in the Buildroot, we could theoretically compile docker-ce and offer it as a package.
The kernel will still need to be compiled in a different way
I see heresome kernel params, but isn't the tricky part to compile the binary?
Hi all,
Just posting a solution I came up with:
Along with a script to build OpenWrt docker images from source:
The makefile in the feed above uses the precompiled static binaries made available by Docker, they are very heavy (130MiB) but avoids the need to try and compile Docker inside the OpenWrt system (which would be nice, but I am not in a position to try it myself).
If someone does try to build Docker/Moby natively, it would be worth seeing if various features can be removed to reduce size, e.g removing swarm, network drivers other than bridge, storage drivers other than overlayfs etc.
As noted above you need to enable most of the "advanced" kernel features (cgroups etc.), I have provided a sample configuration as a reference, and the Docker port forwards will not work so you should treat each container as a client on your OpenWrt network.
Please feel free to collaborate here
2 Likes
Hello. You don't need glibc or docker to run your images. See podman. The most complete list of required options for kernel are listed in check-config.sh. I haven't tried it on openwrt yet, but you shouldn't receive any heavy issues on this way.
1 Like
So should we port podman to OpenWrt instead?
1 Like
Interesting that this got bumped just as I was looking into a few Docker-related items.
The link you posted regarding collaboration isn't really relevant, as that is discussing providing docker images that run OpenWRT in a container. I don't think @mcbridematt understood the goal of the thread either, since that's what he is providing and isn't what the OP was looking for.
This thread started discussing using OpenWRT as a docker engine host - which IS supported in git master and has been for quite a while (first merged in July - https://github.com/openwrt/packages/commits/master/utils/docker-ce ). I just discovered this last week, and strangely, there's NO discussion of this fact anywhere I can find. Any Google search for Docker and OpenWRT is looking at the opposite use case (OpenWRT within a docker container), or using a Docker container to facilitate building OpenWRT on hosts that don't play nice.
I can say that docker-ce running on an aarch64 Raspberry Pi 3b+ runs pretty well. The main issue is that you really need an additional storage device beyond the SD card, at least on pi, because sysupgrade will clobber the partition table, so you can't readily put a third partition on the sd card for persistent storage outside of the rootfs. If you've got Docker images there, it's way too much for sysupgrade to try and back up into RAM when the rootfs gets nuked.
5 Likes
Have a look at the first link - I did exactly that. The 'OpenWrt inside Docker' came in parallel.
I got Docker Engine running by repackaging the prebuilt binaries distributed by Docker (putting the full source build in the 'too hard basket' due to Go and cross-compiling), some months later someone figured out how to build the whole thing from source - hence the docker-ce package - https://github.com/openwrt/packages/pull/8397.
As it turns out the 'slim docker' I was hoping for already exists: https://www.balena.io/engine/ . I haven't had a chance to revisit any of this, I was also hoping to split out runc/containerd/etc so they could be integrated with procd.
1 Like
Sorry to bump this 3-month old thread, but I need to build some lightweight docker containers and it would be great to build them on my OpenWRT router.
I am not a tech guy, and I see there is no official docker-ce packages on the 19.07 repository. Could we expect to get it sometime or is it too difficult to implement docker in OpenWRT?
Thanks in advance and keep up the good work.
1 Like
subscribed
Support for arm cpus was added in November 2019 by juppin https://github.com/openwrt/packages/commit/a6be45ba66 and you are using 19.07 which was branched July 2019 from master.
So no docker for arm on 19.07, you have to use the snapshot builds to get docker on arm.
2 Likes
Thank you.
I have 19.07.03 flashed (May 2020), but I cannot find any docker or docker-ce package at the official repository through opkg.
I have a cortex a9 (arm) processor.
Where should I look for it? I found this:
But I do not know how to install it on my router.
The package doesn't exist in 19.07, it is currently only on master.
2 Likes
Thank you.
I have located the ipk.
docker-ce_19.03.8-2_arm_cortex-a9_vfpv3.ipk
Could I just download it and push it to my 19.07.3 openwrt router?
I have never installed any package other than using the software tab on Luci.
Installing across versions wouldn't be recommended because they are expecting specific system libraries and other things that are built into the OpenWRT version.
1 Like