Docker engine on OpenWrt

Have a look at the first link - I did exactly that. The 'OpenWrt inside Docker' came in parallel.

I got Docker Engine running by repackaging the prebuilt binaries distributed by Docker (putting the full source build in the 'too hard basket' due to Go and cross-compiling), some months later someone figured out how to build the whole thing from source - hence the docker-ce package - https://github.com/openwrt/packages/pull/8397.

As it turns out the 'slim docker' I was hoping for already exists: https://www.balena.io/engine/ . I haven't had a chance to revisit any of this, I was also hoping to split out runc/containerd/etc so they could be integrated with procd.

1 Like

Sorry to bump this 3-month old thread, but I need to build some lightweight docker containers and it would be great to build them on my OpenWRT router.

I am not a tech guy, and I see there is no official docker-ce packages on the 19.07 repository. Could we expect to get it sometime or is it too difficult to implement docker in OpenWRT?

Thanks in advance and keep up the good work.

1 Like

subscribed

Support for arm cpus was added in November 2019 by juppin https://github.com/openwrt/packages/commit/a6be45ba66 and you are using 19.07 which was branched July 2019 from master.
So no docker for arm on 19.07, you have to use the snapshot builds to get docker on arm.

2 Likes

Thank you.

I have 19.07.03 flashed (May 2020), but I cannot find any docker or docker-ce package at the official repository through opkg.

I have a cortex a9 (arm) processor.

Where should I look for it? I found this:

But I do not know how to install it on my router.

The package doesn't exist in 19.07, it is currently only on master.

2 Likes

Thank you.

I have located the ipk.

docker-ce_19.03.8-2_arm_cortex-a9_vfpv3.ipk

Could I just download it and push it to my 19.07.3 openwrt router?

I have never installed any package other than using the software tab on Luci.

Installing across versions wouldn't be recommended because they are expecting specific system libraries and other things that are built into the OpenWRT version.

1 Like

Thank you again.

So my best bet is to wait for a 19.07.4 or 20.XX release to have it included in the stable branch, right?

It seems docker-ce was not included in 19.07.4, I cannot find it in the LuCi package search function.

I hope it will be included in 20.XX

Been waiting for this too.
I've got plenty of headroom on my router though. Using an x86_64. Interested if anyone has used the static binaries from Docker.

Hello, found this thread and is anybody else trying to run the new docker-ce on openWRT?!?
I got a aarch64 host with 2GB of RAM and a 64GB SDCard, so I thought I'd give it a whirl.
docker on x86_64 wouldn't even be worth asking about, that's too easy.

what I installed:

root@n3 opkg list-installed | egrep -i "container|docker|cgroup"
cgroup-tools - 0.41-3
cgroupfs-mount - 2020-06-26-05494281-1
containerd - 1.3.7-1
docker-ce - 19.03.13-1
docker-compose - 1.27.4-1
libcgroup - 0.41-3
luci-app-dockerman - v0.5.13
luci-lib-docker - git-20.211.27004-141857a
lxc-cgroup - 4.0.2-4
python3-docker - 4.3.1-1
python3-dockerpty - 0.4.1-3

yessir, I haz /sys/fs/cgroup mounted:

root@n3 df -hTP
Filesystem           Type            Size      Used Available Capacity Mounted on
/dev/root            ext4           58.8G    468.6M     58.3G   1% /
tmpfs                tmpfs         997.7M    964.0K    996.7M   0% /tmp
tmpfs                tmpfs         512.0K         0    512.0K   0% /dev
/dev/mmcblk0p1       ext4           15.7M     12.1M      3.3M  78% /boot
cgroup               tmpfs         997.7M         0    997.7M   0% sys/fs/cgroup
/dev/root            ext4           58.8G    468.6M     58.3G   1% /opt/docker

yessir, I haz a snapshot release,

because that's all you can do with a rockchip armv8/A53/aarch64 cpu, and docker-ce won't show up in a prod release till later -


root@n3 uname -a
Linux n3 5.4.70 #0 SMP PREEMPT Mon Oct 12 18:59:58 2020 aarch64 GNU/Linux

root@n3 egrep -i "pretty|release" /etc/os-release 
PRETTY_NAME="OpenWrt SNAPSHOT"
OPENWRT_RELEASE="OpenWrt SNAPSHOT r14694-e788e9bd87"

what I did:

root@n3 /etc/init.d/dockerd start

meanwhile chatty little bird rsyslog says:

Oct 14 01:00:07 n3 dockerd-init: Adding docker default interface to network uci config (docker)
Oct 14 01:00:07 n3 dockerd-init: Adding docker default bridge device to network uci config (docker0)
Oct 14 01:00:07 n3 dockerd-init: Adding docker default firewall zone to firewall uci config (docker)
Oct 14 01:00:07 n3 dockerd-init: Updating network uci config option "172.18.0.1/24" for docker default bridge (docker0)
Oct 14 01:00:08 n3 dockerd-init: Updating network uci config option "172.18.0.1/24" for docker default bridge (docker0)
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.081074086Z" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
Oct 14 01:00:09 n3 modprobe: failed to find a module named aufs
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.094011984Z" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577416251Z" level=warning msg="Your kernel does not support swap memory limit"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577614309Z" level=warning msg="Your kernel does not support cgroup blkio weight"
Oct 14 01:00:09 n3 dockerd[1898]: time="2020-10-14T01:00:09.577653978Z" level=warning msg="Your kernel does not support cgroup blkio weight_device"
Oct 14 01:00:10 n3 dockerd[1898]: failed to start daemon: Devices cgroup isn't mounted

also:

opkg list | egrep -i "aufs|devmap"

whattheheck is aufs? devmapper?

we aint got no multipathd and lvm around here. This is a single SDCard rootdisk! Not even SPI/eMMC/flash!

Do I haveta supply arcane cgroups kernel boot args like we do in CentOS-land?

as things slip way from real docker to moby to podman?

There is currently some work happening to get podman to work with uxc, which basically replaces Docker with a much lower footprint.

3 Likes

Try configuring docker to use overlay2 instead of aufs, it's the newer and better storage method.

wow, very interesting! just to let you know, the PR you mentioned has been merged now.

I had the same issues with a snapshot pi build.... took a long time to figure out and dont know why.... but the fix for me was

Update cgroup startup - comment out section

vi /etc/init.d/cgroupfs-mount

#        if mountpoint -q /sys/fs/cgroup; then
#                umount /sys/fs/cgroup/
#        fi

update dockerd start order from 25 -> 60

vi /etc/init.d/dockerd

        START=60

Update startup to setup tmpfs

        mount -t tmpfs -o size=32k tmpfs /run

If no longer needed then great but if stuck, try the above....

2 Likes

Done on MVEBU espressobin-ultra wih OpenWrt 21.02-RC3 and now, docker no more do the folowing error :

Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

thanks for the tips

1 Like

TRY with SUCCESS on 21.02.0 (and update of dockerd docker docker-compose luci-dockerman)
MVEBU ARM64 EspressoBin-Ultra...

root@ultra:~# cat /etc/config/dockerd
# The following settings require a restart of docker to take full effect, A reload will only have partial or no effect:
# bip
# blocked_interfaces
# extra_iptables_args
# device

config globals 'globals'
#	option alt_config_file '/etc/docker/daemon.json'
	option enable '1'
	option data_root '/srv/docker/'
#	option data_root '/opt/docker/'
	option log_level 'warn'
	option iptables '1'
	list hosts 'unix:///var/run/docker.sock'
#	option bip '172.18.0.1/24'
#	option fixed_cidr '172.17.0.0/16'
#	option fixed_cidr_v6 'fc00:1::/80'
#	option ipv6 '1'
#	option ip '::ffff:0.0.0.0'
#	list dns '172.17.0.1'
#	list registry_mirrors 'https://<my-docker-mirror-host>'
	list registry_mirrors 'https://hub.docker.com'
	option remote_endpoint '0'
#	option bridge 'br-container'

# Docker ignores fw3 rules and by default all external source IPs are allowed to connect to the Docker host.
# See https://docs.docker.com/network/iptables/ for more details.
# firewall config changes are only additive i.e firewall will need to be restarted first to clear old changes,
# then docker restarted to load in new changes.
config firewall 'firewall'
	option device 'docker0'
	list blocked_interfaces 'wan'
#	option extra_iptables_args '--match conntrack ! --ctstate RELATED,ESTABLISHED' # allow outbound connections
root@ultra:~# cat /root/LMS/docker-compose.yml 
version: '3.5'
volumes:
  gmusic:
	driver: local
	driver_opts:
	  type: nfs
	  o: "addr=10.4.2.25,nolock,soft,rw"
	  device: ":/GDATA02/MUSIQUES"
services:
  lms:
	container_name: lms
	image: lmscommunity/logitechmediaserver:stable
	volumes:
	  - /srv/SHARE/LMS/CONFIG:/config:rw
	  - /srv/SHARE/MUSIC:/music:ro
	  - /srv/SHARE/LMS/PLAYLIST:/playlist:rw
	  - /etc/localtime:/etc/localtime:ro
	  - /etc/timezone:/etc/TC:ro
	  - gmusic:/gmusic:ro
#    ports:
#      - 9002:9002/tcp
#      - 9090:9090/tcp
#      - 3483:3483/tcp
#      - 3483:3483/udp
	restart: always
	environment:
	  - HTTP_PORT=9000
	  - PUID=1000
	  - PGID=1000
	network_mode: host
root@ultra:~# docker-compose -f /root/LMS/docker-compose.yml up --no-start
Building with native build. Learn about native build in Compose here: https://docs.docker.com/go/compose-native-build/
Recreating lms ... done

FIX : I had to create a missing file which link to a non existing /tmp/localtime

touch /etc/localtime

Then... http://ultra:9000 and LMS is UP and Works fine (look to be faster than with LXC)

Screenshot 2021-10-01 at 23-21-33 BERMUDES - FAUVE Logitech Media Server

1 Like

Hi. When downloading an image, I get " Bad GatewayThe process did not produce any response"
What to fix?

This was immensely helpful on the latest 23.05.0.

Using prior versions I did not need to do anything other than a screen in rc.local however I was forced to use:

#!/bin/sh /etc/rc.common
START=60
STOP=80
boot() {
        mount -t tmpfs -o size=32k tmpfs /run
        screen -m -d -S docker sh -c "dockerd --ip-masq=false --iptables=false --data-root=/mnt/nvme_shares1/docker"
}

start() {
        screen -m -d -S docker sh -c "dockerd --ip-masq=false --iptables=false --data-root=/mnt/nvme_shares1/docker"
}

stop() {
        screen -X -S "docker" quit
}