I see heresome kernel params, but isn't the tricky part to compile the binary?
Hi all,
Just posting a solution I came up with:
Along with a script to build OpenWrt docker images from source:
The makefile in the feed above uses the precompiled static binaries made available by Docker, they are very heavy (130MiB) but avoids the need to try and compile Docker inside the OpenWrt system (which would be nice, but I am not in a position to try it myself).
If someone does try to build Docker/Moby natively, it would be worth seeing if various features can be removed to reduce size, e.g removing swarm, network drivers other than bridge, storage drivers other than overlayfs etc.
As noted above you need to enable most of the "advanced" kernel features (cgroups etc.), I have provided a sample configuration as a reference, and the Docker port forwards will not work so you should treat each container as a client on your OpenWrt network.
Hello. You don't need glibc or docker to run your images. See podman. The most complete list of required options for kernel are listed in check-config.sh. I haven't tried it on openwrt yet, but you shouldn't receive any heavy issues on this way.
1 Like
So should we port podman to OpenWrt instead?
1 Like
Interesting that this got bumped just as I was looking into a few Docker-related items.
The link you posted regarding collaboration isn't really relevant, as that is discussing providing docker images that run OpenWRT in a container. I don't think @mcbridematt understood the goal of the thread either, since that's what he is providing and isn't what the OP was looking for.
This thread started discussing using OpenWRT as a docker engine host - which IS supported in git master and has been for quite a while (first merged in July - https://github.com/openwrt/packages/commits/master/utils/docker-ce ). I just discovered this last week, and strangely, there's NO discussion of this fact anywhere I can find. Any Google search for Docker and OpenWRT is looking at the opposite use case (OpenWRT within a docker container), or using a Docker container to facilitate building OpenWRT on hosts that don't play nice.
I can say that docker-ce running on an aarch64 Raspberry Pi 3b+ runs pretty well. The main issue is that you really need an additional storage device beyond the SD card, at least on pi, because sysupgrade will clobber the partition table, so you can't readily put a third partition on the sd card for persistent storage outside of the rootfs. If you've got Docker images there, it's way too much for sysupgrade to try and back up into RAM when the rootfs gets nuked.
5 Likes
Have a look at the first link - I did exactly that. The 'OpenWrt inside Docker' came in parallel.
I got Docker Engine running by repackaging the prebuilt binaries distributed by Docker (putting the full source build in the 'too hard basket' due to Go and cross-compiling), some months later someone figured out how to build the whole thing from source - hence the docker-ce package - https://github.com/openwrt/packages/pull/8397.
As it turns out the 'slim docker' I was hoping for already exists: https://www.balena.io/engine/ . I haven't had a chance to revisit any of this, I was also hoping to split out runc/containerd/etc so they could be integrated with procd.
1 Like
Sorry to bump this 3-month old thread, but I need to build some lightweight docker containers and it would be great to build them on my OpenWRT router.
I am not a tech guy, and I see there is no official docker-ce packages on the 19.07 repository. Could we expect to get it sometime or is it too difficult to implement docker in OpenWRT?
Thanks in advance and keep up the good work.
1 Like
subscribed
Support for arm cpus was added in November 2019 by juppin https://github.com/openwrt/packages/commit/a6be45ba66 and you are using 19.07 which was branched July 2019 from master.
So no docker for arm on 19.07, you have to use the snapshot builds to get docker on arm.
2 Likes
Thank you.
I have 19.07.03 flashed (May 2020), but I cannot find any docker or docker-ce package at the official repository through opkg.
I have a cortex a9 (arm) processor.
Where should I look for it? I found this:
But I do not know how to install it on my router.
The package doesn't exist in 19.07, it is currently only on master.
2 Likes
Thank you.
I have located the ipk.
docker-ce_19.03.8-2_arm_cortex-a9_vfpv3.ipk
Could I just download it and push it to my 19.07.3 openwrt router?
I have never installed any package other than using the software tab on Luci.
Installing across versions wouldn't be recommended because they are expecting specific system libraries and other things that are built into the OpenWRT version.
1 Like
Thank you again.
So my best bet is to wait for a 19.07.4 or 20.XX release to have it included in the stable branch, right?
It seems docker-ce was not included in 19.07.4, I cannot find it in the LuCi package search function.
I hope it will be included in 20.XX
Been waiting for this too.
I've got plenty of headroom on my router though. Using an x86_64. Interested if anyone has used the static binaries from Docker.
Hello, found this thread and is anybody else trying to run the new docker-ce on openWRT?!?
I got a aarch64 host with 2GB of RAM and a 64GB SDCard, so I thought I'd give it a whirl.
docker on x86_64 wouldn't even be worth asking about, that's too easy.
what I installed:
root@n3 opkg list-installed | egrep -i "container|docker|cgroup"
cgroup-tools - 0.41-3
cgroupfs-mount - 2020-06-26-05494281-1
containerd - 1.3.7-1
docker-ce - 19.03.13-1
docker-compose - 1.27.4-1
libcgroup - 0.41-3
luci-app-dockerman - v0.5.13
luci-lib-docker - git-20.211.27004-141857a
lxc-cgroup - 4.0.2-4
python3-docker - 4.3.1-1
python3-dockerpty - 0.4.1-3
yessir, I haz /sys/fs/cgroup mounted:
root@n3 df -hTP
Filesystem Type Size Used Available Capacity Mounted on
/dev/root ext4 58.8G 468.6M 58.3G 1% /
tmpfs tmpfs 997.7M 964.0K 996.7M 0% /tmp
tmpfs tmpfs 512.0K 0 512.0K 0% /dev
/dev/mmcblk0p1 ext4 15.7M 12.1M 3.3M 78% /boot
cgroup tmpfs 997.7M 0 997.7M 0% sys/fs/cgroup
/dev/root ext4 58.8G 468.6M 58.3G 1% /opt/docker
yessir, I haz a snapshot release,
because that's all you can do with a rockchip armv8/A53/aarch64 cpu, and docker-ce won't show up in a prod release till later -
root@n3 uname -a
Linux n3 5.4.70 #0 SMP PREEMPT Mon Oct 12 18:59:58 2020 aarch64 GNU/Linux
root@n3 egrep -i "pretty|release" /etc/os-release
PRETTY_NAME="OpenWrt SNAPSHOT"
OPENWRT_RELEASE="OpenWrt SNAPSHOT r14694-e788e9bd87"
what I did:
root@n3 /etc/init.d/dockerd start
meanwhile chatty little bird rsyslog says:
Oct 14 01:00:07 n3 dockerd-init: Adding docker default interface to network uci config (docker)
Oct 14 01:00:07 n3 dockerd-init: Adding docker default bridge device to network uci config (docker0)
Oct 14 01:00:07 n3 dockerd-init: Adding docker default firewall zone to firewall uci config (docker)
Oct 14 01:00:07 n3 dockerd-init: Updating network uci config option "172.18.0.1/24" for docker default bridge (docker0)
Oct 14 01:00:08 n3 dockerd-init: Updating network uci config option "172.18.0.1/24" for docker default bridge (docker0)
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.081074086Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Oct 14 01:00:09 n3 modprobe: failed to find a module named aufs
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.094011984Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577416251Z" level=warning msg="Your kernel does not support swap memory limit"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577614309Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577653978Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Oct 14 01:00:10 n3 dockerd[1898]: failed to start daemon: Devices cgroup isn't mounted
also:
opkg list | egrep -i "aufs|devmap"
whattheheck is aufs? devmapper?
we aint got no multipathd and lvm around here. This is a single SDCard rootdisk! Not even SPI/eMMC/flash!
Do I haveta supply arcane cgroups kernel boot args like we do in CentOS-land?
as things slip way from real docker to moby to podman?
There is currently some work happening to get podman to work with uxc, which basically replaces Docker with a much lower footprint.
3 Likes
Try configuring docker to use overlay2 instead of aufs, it's the newer and better storage method.