However, it's not quite working. I think the issue is how I'm setting up the container networking. I have 2 dedicated NICs: WAN (that is directly connected to the fibre ONT) and a LAN nic that is plugged into a WiFi switch/AP.
My openwrt.conf file looks as follow:
Hi @gmiranda, I haven't tested this with PPPoE at all as my ISP doesn't use it.
The issue is probably the same as the one described above by @markthehipster. My guess is that you will have to manually load the PPPoE kernel modules on your host (not inside the OpenWrt container).
Hi @oofnik, thanks for all your hard work.
I am having trouble on my Raspberry Pi CM4 trying to run "make build" - for some reason it doesn't parse the square brackets in the build.sh, which is really strange as it seems like perfectly good bash to me.
I tried running chmod +x openwrt.config && source openwrt.config && ./build.sh on it's own and it gives the same fault
I tried finding the appropriate commands and stuffing them with the correct variables, but it still doesn't work.
Docker works fine with other apps. I installed domoticz and it worked fine.
Kernel uname -a Linux raspberrypi 5.10.17-v8+ #1403 SMP PREEMPT Mon Feb 22 11:37:54 GMT 2021 aarch64 GNU/Linux
Its running latest Raspbian but its a bit of a pet with a few hacks here and there, but nothing related to bash. It is running on a 128GB PCIe NVME SSD (which is amazing btw).
Can you shed any light on why the build script might be failing like this?
See below:
root@raspberrypi:/home/pi/docker-openwrt# make build
./build.sh
./build.sh: 29: ./build.sh: [[: not found
./build.sh: 34: ./build.sh: [[: not found
./build.sh: 37: ./build.sh: [[: not found
./build.sh: 40: ./build.sh: [[: not found
Unsupported architecture!
./build.sh: 71: ./build.sh: [[: not found
make: *** [Makefile:7: build] Error 1
root@raspberrypi:/home/pi/docker-openwrt#
I tried the following:
source openwrt.conf
tmpdir=$(mktemp -u -p .)
rootfs_url="https://downloads.openwrt.org/releases/${OPENWRT_SOURCE_VER}/targets/armvirt/64/openwrt-${OPENWRT_SOURCE_VER}-armvirt-64-default-rootfs.tar.gz"
version="https://downloads.openwrt.org/releases/${OPENWRT_SOURCE_VER}/targets/armvirt/64/version.buildinfo"
wget "${rootfs_url}" -O rootfs.tar.gz
wget "${version}" -O version.buildinfo
docker build --build-arg ts="$(date)" --build-arg version="$(cat version.buildinfo)" -t $IMAGE:$TAG .
Sending build context to Docker daemon 2.533MB
Step 1/13 : FROM scratch
--->
Step 2/13 : ADD rootfs.tar.gz /
---> 2c3ec144e456
Step 3/13 : RUN mkdir -p /var/lock
---> Running in 038e826503a1
The command '/bin/sh -c mkdir -p /var/lock' returned a non-zero code: 159
Bash version:
root@raspberrypi:/home/pi/docker-openwrt# bash --version
GNU bash, version 5.0.3(1)-release (arm-unknown-linux-gnueabihf)
Copyright (C) 2019 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
The issue is that I didn't test the build script in enough environments - it works on Alpine / busybox sh but not on Debian / Ubuntu sh. It can be fixed temporarily by modifying the shebang from #!/bin/sh to #!/usr/bin/env bash to force the script to use bash instead.
I'll have a closer look at it later today to fix the issue once and for all.
Thanks @oofnik,
Yep I discovered that is was using /bin/sh after I posted the message above, and managed to fix that on my own. I still get the Error 159 after running make build however. Then make run complains that there is no build.
I also tried just running docker run -it --name openwrt openwrtorg/rootfs:latest, which fails with standard_init_linux.go:207: exec user process caused "exec format error"
Also tried docker pull oofnik/openwrt:latest, which threw docker: Error response from daemon: manifest for oofnik/openwrt:latest not found.
exec format error happens because you are trying to run an amd64 Docker image on a armhf platform (or aarch64 if you are running a 64-bit kernel). Docker images must match your hardware's CPU architecture.
Have a look at this doc to figure out which image you can run on your RPi. I'll go ahead and upload the latest 19.07.7 image shortly.
Thanks for your tips. I've ended up going the LXD route (using linuxcontainers.org's image) and seems to be working really fine, with the container even being able to load kernel modules.
Thanks @oofnik. I had my openwrt.conf file set to armvirt-64. I have the 64 bit kernel in Raspbian but it doesn't appear to work. As soon as I changed it to armvirt-32, it started working! Any idea how I can make it support aarch64? Do I need to install Raspbian x64 beta?
Further research:
It's not simply the case to have 64 bit support in the kernel, apparently you need 64 bit userland as well. I got it working under nspawn after a LOT of faffing. Follow this: Using Docker Inside the Container · sakaki-/raspbian-nspawn-64 Wiki (github.com). Be prepared for it not to work first-time and you need to reboot your Pi before it does...
I am planning to run openwrt in a docker with multi wan. My host OS is Ubuntu 20.04.
My Host OS shows : 1 Intel Card, 1 realtek Card and 1 linksys USB 2 Ethernet adapter.
Following your instruction @oofnik I am able to make 1 NIC as WAN and 1 as LAN. I have disabled wifi.
How to access the 3rd LAN card in openwrt and configure it as 2nd WAN. This is my first hands on with docker. Can you please help.
Had anyone tried this on an Odroid N2? I am
Hoping to give it a go.
Someone mentioned earlier that OpenWrt is not designed to run in a containerised environment but that makes me think: are there any open source router firmwares upcoming which are developed for the sole purpose of running in Docker?
I think the main problem is the docker networks. I would use one docker network as management network and then connect to other networks without using the docker api.
Can anybody explain the difference between setting bridge or macvlan for LAN_DRIVER option in docker.conf? My Docker host machine (running Openmediavault+Portainer) has 2 ethernet ports so I assigned the 1st ethernet port as LAN_PARENT with LAN_DRIVER=macvlan (instead of default bridge) and the 2nd ethernet port as WAN_PARENT. OpenWRT is running and I can connect to the internet OK, but now I can't figure out how to access the host machine's GUI or its exposed Samba/NFS shares anymore...
This is really cool. Since I now have the internet plugged directly into my host server (kinda scary) are there any firewall rules i need to run on the host machine to prevent having the host exposed directly to the WAN? I'm using 2 nics (WAN and LAN).
I'm guessing since i'm only getting an wan IP address on the macvlan it's only exposed in the openwrt docker container.
OpenWrt comes with a default set of firewall rules that should be sufficiently sane for a typical home setting. You are correct that your host is not directly exposed to internet traffic as long as it does not have a public IP address with the exception of broadcast traffic. You can run a tcpdump on the interface to confirm this. Depending on your WAN link type, as an example you may see your neighbor's CPE DHCP requests (or more, if something is misconfigured).
The only other concern is some kind of exploit in the Docker engine that allows breaking out of the containerized environment. As long as you keep your OS and Docker engine reasonably up to date and keep your passwords secure, you should sleep soundly at night.
@oofnik is there any way to keep the default docker bridge working in it's own subnet?
After i run the startup script the other docker containers that are running can no longer connect to the internet. If i restart them with
--netowrk openwrt-lan
They will connect to the internet and get an DHCP IP address from openwrt.
But is there any way to make the default docker bridge route to the internet correctly running in it's own private subnet.
This is with 2 nics (macvlan on WAN and bridge on LAN).
Just thinking out loud, I'm probably wrong, but I think everything on the host goes out to the internet through the openwrt-lan Bridge, and I don't think you can make the docker0 bridge use a openwrt-lan bridge as a gateway.
UPDATE: I changed the LAN interface from bridge to macvlan and now it works.
Hi all, I didn't read through all the thread but since I'm currently maintaining the official Docker images I'd be keen to get some ideas on how to improve them. Some time ago I send an email over the openwrt-devel mailing list with very little response, please share your opinions here on what I can do.
The wiki, it's horribly outdated, it makes it seem like it's not possible to do. After reading the wiki i was just going to forget about even trying it. It should probably be deleted or just redirected to this thread.
I also think a step by step write up of how to do a simple setup with 2 nics and the commandline steps to setup the docker networks (basically a step by step of what the this script is doing minus the wifi with explainations) on the dockerhub page would go a long way. It's actually not that complicated. And you could just redirect the wiki there.