OpenWrt Forum Archive

Topic: running OpenWRT as a LXD/LXC container

The content of this topic has been archived on 2 May 2018. There are no obvious gaps in this topic, but there may still be some posts missing at the end.

Here's how I created an LXD/LXC container running OpenWRT:

I used my Ubuntu 16.04 desktop as a host, with lxd installed and configured
(apt-get install lxd; lxd init).  You can also use an AWS Ubuntu 16.04 image, or an OpenStack KVM VPS, but you will need some extra networking steps to access the container from outside (e.g. iptables or an HTTP reverse proxy).

Download the latest openwrt rootfs.tar.gz for x86_64 platform
I used: http://downloads.openwrt.org/chaos_calm … tfs.tar.gz

create a metadata.yaml file with the contents:

{
    "architecture": "x86_64",
    "creation_date": 1472916664,
    "properties": {
        "architecture": "x86_64",
        "description": "openwrt 15.05.1 x86_64 (default) (20160903_18:31)",
        "name": "openwrt-15.05.1-x86_64-default-20160903_18:31",
        "os": "openwrt",
        "release": "15.0.1",
        "variant": "default"
    }
}

Obviously you can update the times.  I use a script to generate this file with the current time.


# create a metadata tarball
tar cfz  meta.tar.gz metadata.yaml

# import an image with the metadata and rootfs
lxc image import meta.tar.gz openwrt-15.05.1-x86-64-rootfs.tar.gz

# add an alias to the newly created image:
lxc image alias create openwrt <fingerprint>  # use the fingerpring printed by the import

# Now create a container.  It needs to have privileged permissions:
lxc launch openwrt --config security.privileged=true t1  # "t1" is the name of the container

# Start a shell in the container:
lxc exec t1 ash

BusyBox v1.23.2 (2016-01-02 12:56:51 CET) built-in shell (ash)

# ps

    1 root     15136 S    /sbin/procd
   38 root      9296 S    ash
   48 root      9292 R    ps

Not much is running.  It seems to have been partially booted.  We'll fix it below, but first, configure the network to use the LXD DHCP server:

edit /etc/config/network and replace the 'lan' interface with:

config interface 'lan'
    option ifname 'eth0'
    option proto 'dhcp'

Now run the following mknod commands.  It's a good idea to save them in a script, because you will need to run them every time the container boots:

mknod -m 666 /dev/zero c 1 5
mknod -m 666 /dev/full c 1 7
mknod -m 666 /dev/random c 1 8
mknod -m 666 /dev/urandom c 1

I have no idea why this works.  I found these lines in lxc/templates/ in the github lxc code.
Running mknod is the reason the container needs privileged permissions.

That's it.  The container now runs.  You can reboot or halt (which you could not do without the missing devices).

You can find the container's ip address (10.x.x.x) and http to its web interface.

You should also be able to access the internet from the container.

Some things don't work yet:

* rebooting the container loses the devices we created.  We need to run mknod again.
I tried putting this in rc.local or /etc/preinit, but it didn't work.

* ssh into the container gives this error:
PTY allocation request failed on channel 0
shell request failed on channel 0

Perhaps someone more knowledgeable can fill-in the missing pieces.

(Last edited by votsalo on 4 Sep 2016, 20:48)

As seen from some dmesg output (below), OpenWRT cannot mount some filesystems (including /dev).
This may be fixable by configuring the container in LXD to allow this.  I don't know how to do this yet.
I tried disabling apparmor, but it made no difference.
I also tried launching the container with these options, but it didn't help:

lxc launch openwrt-chaos_calmer -c security.privileged=true -c security.nesting=true t1

relevant messages from dmesg:

[ 9374.063538] audit: type=1400 audit(1473064773.886:60): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-b1_</var/lib/lxd>" name="/dev/pts/" pid=15546 comm="init" fstype="devpts" srcname="devpts" flags="rw, noatime"
[ 9376.087335] audit: type=1400 audit(1473064775.910:61): apparmor="DENIED" operation="mount" info="failed flags match" error=-13 profile="lxd-b1_</var/lib/lxd>" name="/" pid=15622 comm="mount_root" flags="rw, remount, noatime"
[ 9376.109758] audit: type=1400 audit(1473064775.934:62): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-b1_</var/lib/lxd>" name="/dev/pts/" pid=15546 comm="procd" fstype="devpts" srcname="devpts"
[ 9386.050009] audit: type=1400 audit(1473064785.874:63): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-b1_</var/lib/lxd>" name="/proc/sys/" pid=15961 comm="umount" flags="ro, remount"
[ 9386.065568] audit: type=1400 audit(1473064785.890:64): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-b1_</var/lib/lxd>" name="/proc/" pid=15961 comm="umount" flags="ro, remount"


[ 8344.603832] audit: type=1400 audit(1473063744.416:41): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-openwrt_</var/lib/lxd>" name="/dev/" pid=12377 comm="umount" flags="ro, remount"
[ 8344.603975] audit: type=1400 audit(1473063744.416:42): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-openwrt_</var/lib/lxd>" name="/tmp/" pid=12377 comm="umount" flags="ro, remount"
[ 8344.801838] audit: type=1400 audit(1473063744.616:43): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-openwrt_</var/lib/lxd>" name="/proc/sys/" pid=12377 comm="umount" flags="ro, remount"
[ 8344.813281] audit: type=1400 audit(1473063744.628:44): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-openwrt_</var/lib/lxd>" name="/proc/" pid=12377 comm="umount" flags="ro, remount"
[ 8344.813392] audit: type=1400 audit(1473063744.628:45): apparmor="DENIED" operation="mount" info="failed type match" error=-13 profile="lxd-openwrt_</var/lib/lxd>" name="/dev/" pid=12377 comm="umount" flags="ro, remount"

The discussion might have continued from here.