OpenWrt ARM in Linux container

I want to run OpenWrt on an Odroid-C2 as a wired router. Odroid-C2 uses SoC Amlogic S905 - Quad Core Cortex™-A53 1.5GHz 64bit ARMv8 processor.

It has one gigabit port, and a second can be added via USB which is supposed to be limited to about 300 mbit (source).

To date there is no suitable target, but efforts has been made to encourage the creation of support for Amlogic S9xxx SoC family

Another option is to run OpenWrt through Linux containers. If you compare network type options provided by Linux Containers (LXC) and Docker many options are similar, however one option seems exclusive for LXC and that is support for the network type "phys".

The "[i].type" setting is used to

specify what kind of network virtualization to be used for the container". And "phys" is described as "phys: an already existing interface specified by the[i].link is assigned to the container.

Since running OpenWrt on the Odroid box is the only thing I want to use it for, I assume the "phys" network type may be a good fit for passing all network traffic into the container where I plan to run OpenWrt. Also, it appears to be support for running LXC using Odroid as detailed in this article. The article was dated late 2015, since then the host OS, Ubuntu, has evolved to ubuntu-18.04-3.16-minimal-odroid-c2-20180626.img, and there are work going (inofficial) for providing Odroid C2 mainline kernel support.

To run OpenWrt in LXC I need it compiled for the ARM architecture.Using the default build instructions, I would have needed something similar to the target "x86\generic target", but there there is no such option for ARM. Closest I can find is a target aimed for "QEMU ARM virt machine".

Once I figure out how to build OpenWrt for the ARM architecture, I hope to get inspiration from the work done over at how to build LXD images from OpenWrt rootfs tarballs

Is LXC network type "phys" a good option?

How to compile OpenWrt to fit my needs (ARMv8 64 bit)?

Is something else I need to know to get forward?


I doubt you'll find it very useful until kernel 4.19 lands even then Amlogic support seems quite spotty.

Thanks for sharing!

The qemu builds work pretty good.... although i'm not aware of their production iostats.....

well worth trying...... while something more polished hits the scene.......

you should be able to run arm and x86 targets within their respective interpreters.... ( qemu binaries )....

Decided to shut down the "container initiative" when I learned that early work has been done to add support for an (openwrt) target referred to as "mesongx".

However, I can´t risk the mental health of my family by using pretty untested software for providing access to internet. Hence I´ll have to wait until the support matures for the mesongx target.

As for now I´m releasing my Haswell NUC of its media-center duties, swapping it with this Odroid-C2 targeting Thought about adding an additional USB NIC to the NUC and run a virtual firewall (Opnsense etc..) in ESXI, but you have to resort to unofficial drivers to get USB NICs working in ESXI. Stupid risks. Again. I simply have to force myself to always decide on robust solutions for this crucial piece of functionality such as the internet connection.

Anyhow, thanks all for your feedback!

Hi all,

I know, old thread

I am using an OpenWrt 18.06 lxc container on an Odroid C1 as my internet access router. In front of the Odroid C1 a managed switch is used the combine the incomming stream from my provider and the outgoing streams ( lan and guest network ) and pump it via different VLAN into the Openwrt Container. Working without any problem. The base system on the Odroid C1 is a Debian Stretch System.


Hi powo01,

I'm glad you posted, because I have a great deal of interest in achieving something similar to you, on Ubuntu 18.04 server running on a raspberry pi 3b+!

Would you mind if I asked some questions about how you set your container up, as I'm relatively new to lxc and keep having issues with the procd process stalling, due to issues with mounting?

I've mostly followed the following tutorial, link, adjusting certain things to suit my requirements, such as the configuration of the interfaces.

I find that the script provided doesn't work, returning an error message:

mount: mounting devtmpfs on /dev failed: Permission denied
waiting for rest of boot up...

which just goes on until I lxc stop -f container.

When I start the container, I get the following message to stdout on the host's prompt (but not on prompts through ssh), followed by a stream of continuous lines:

/etc/preinit: line 1: can't create /dev/kmsg: Operation not permitted
open: No such file or directory
open: No such file or directory
open: No such file or directory

which, again, just goes on until I stop the container.

lxc info --show-log openwrt produces only one line of output:

lxc openwrt 20190512114614.153 WARN     conf - conf.c:lxc_setup_devpts:1616 - Invalid argument - Failed to unmount old devpts instance

Kind regards,

My setup uing the raw lxc tools and not the lxd setup from your referenced tuturial

can you show you lxc log file which is created during the lxc-start startup in the container root directory ?

My lxc config file is nothing special = veth = 2000 = up = br0 = xx:xx:xx:xx:xx = = lan0 = phys = up = ifb0 = ifb0 = phys = up = vlan4 = guest = phys = up = vlan2 = fibre

lxc.rootfs = /var/lib/lxc/base_images/openwrt-18.06.0
lxc.utsname = openwrt

lxc.hook.mount = /var/lib/lxc/openwrt-chaos_calmer/

lxc.tty = 1
lxc.pts = 4

lxc.mount = /var/lib/lxc/openwrt-chaos_calmer/fstab
lxc.arch = armv6l

# Permanently tweaked resource settings
lxc.cgroup.memory.limit_in_bytes = 32M
lxc.cgroup.memory.memsw.limit_in_bytes = 48M
lxc.cgroup.cpuset.cpus = 3

lxc.cgroup.devices.deny = a
# Allow any mknod (but not using the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
#lxc.cgroup.devices.allow = c 4:0 rwm
#lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
lxc.cgroup.devices.allow = c 10:229 rwm
lxc.cgroup.devices.allow = c 10:200 rwm
lxc.cgroup.devices.allow = c 1:7 rwm
lxc.cgroup.devices.allow = c 10:228 rwm
#lxc.cgroup.devices.allow = c 108:0 rwm
lxc.cgroup.devices.allow = c 10:57 rwm
lxc.cgroup.devices.allow = c 252:0 rwm = 0
lxc.start.delay = 10

lxc.aa_profile = unconfined

the Openwrt image in /var/lib/lxc/base_images/openwrt-18.06.0 directory is a matching, in my case ARM Cortex A5, unpacked image. I using three interfaces fibre, guest and lan ( without bridging )


Hi Powo01,

Sorry for the gap in replying, I had a weird issue where the firmware for the onboard broadcomm wifi chip stopped working, which effectively halted me working any further on this...

Yeah, if anyone else is interested in replicating this setup, I found the underlying cause for the problem, via this forum topic (link).

Apparently, the issue is that the openwrt image - not realising it's in a container - tries to mount /dev/* itself, thus over-mounting the /dev* setup by LXD. stgraber provides the solution, which is simply to unmount the second mount:

umount -l /dev

which fixes everything, no need to use mknod .

Someone has made a build script for building openwrt images suitable for lxc, although it doesn't have support form aarch32 yet. You can checkout the patch he uses that stops procd from trying to mount /dev/* here though (link).

edit: It seems this got picked up by the people maintaining procd, who have made changes to detect when running in a container (link) and modify behaviour accordingly, so hopefully just doing a fresh image build should provide a container-friendly image.

edit2: And trying a freshly compiled image, ls /dev gives

console    loop106    loop12     loop133    loop147    loop160    loop174    loop188    loop200    loop214    loop228    loop241    loop255    loop39     loop52     loop66     loop8      loop93     pts
fd         loop107    loop120    loop134    loop148    loop161    loop175    loop189    loop201    loop215    loop229    loop242    loop26     loop4      loop53     loop67     loop80     loop94     random
full       loop108    loop121    loop135    loop149    loop162    loop176    loop19     loop202    loop216    loop23     loop243    loop27     loop40     loop54     loop68     loop81     loop95     shm
fuse       loop109    loop122    loop136    loop15     loop163    loop177    loop190    loop203    loop217    loop230    loop244    loop28     loop41     loop55     loop69     loop82     loop96     stderr
kmsg       loop11     loop123    loop137    loop150    loop164    loop178    loop191    loop204    loop218    loop231    loop245    loop29     loop42     loop56     loop7      loop83     loop97     stdin
log        loop110    loop124    loop138    loop151    loop165    loop179    loop192    loop205    loop219    loop232    loop246    loop3      loop43     loop57     loop70     loop84     loop98     stdout
loop0      loop111    loop125    loop139    loop152    loop166    loop18     loop193    loop206    loop22     loop233    loop247    loop30     loop44     loop58     loop71     loop85     loop99     tty
loop1      loop112    loop126    loop14     loop153    loop167    loop180    loop194    loop207    loop220    loop234    loop248    loop31     loop45     loop59     loop72     loop86     lxd        urandom
loop10     loop113    loop127    loop140    loop154    loop168    loop181    loop195    loop208    loop221    loop235    loop249    loop32     loop46     loop6      loop73     loop87     mmcblk0    zero
loop100    loop114    loop128    loop141    loop155    loop169    loop182    loop196    loop209    loop222    loop236    loop25     loop33     loop47     loop60     loop74     loop88     mmcblk0p1
loop101    loop115    loop129    loop142    loop156    loop17     loop183    loop197    loop21     loop223    loop237    loop250    loop34     loop48     loop61     loop75     loop89     mmcblk0p2
loop102    loop116    loop13     loop143    loop157    loop170    loop184    loop198    loop210    loop224    loop238    loop251    loop35     loop49     loop62     loop76     loop9      mqueue
loop103    loop117    loop130    loop144    loop158    loop171    loop185    loop199    loop211    loop225    loop239    loop252    loop36     loop5      loop63     loop77     loop90     net
loop104    loop118    loop131    loop145    loop159    loop172    loop186    loop2      loop212    loop226    loop24     loop253    loop37     loop50     loop64     loop78     loop91     null
loop105    loop119    loop132    loop146    loop16     loop173    loop187    loop20     loop213    loop227    loop240    loop254    loop38     loop51     loop65     loop79     loop92     ptmx

right off the bat.