How to recover space in a 4/32 device in order to install wireguard client?

Cyborg · March 9, 2023, 4:52am

Hi guys

First of all: I just discovered OpenWRT and I want to extend my congratulations to the OpenWRT's developers, you guys do a really AMAZING work!!! With that said...

I own a TP-Link TL-WR841N/ND v11 Wi-Fi Router to which I recently installed the latest available (for my device) OpenWrt 18.06.9 r8077-7cbbab7246 / LuCI openwrt-18.06 branch (git-20.319.49209-ab22243). I did read the 4/32 devices warning before proceed with the installation but to be completly honest, I have not experienced any bug so far.

The thing is: I'm using my router to get the internet from a Wi-Fi as client and share that internet connection to the LAN and to a HotSpot. So good, so far. I live in Cuba, and due to US embargo laws, I can't access to dockerhub container registry
I do have access to a AWS account where I can easily start a WireGuard server and connect my router to that VPN server and then route all the traffic trough the VPN interface.

The thing is... I'm hitting the wall of the 4/32 devices, not because of the RAM but because of the space. The following picture shows a nearly fresh OpenWRT install on my router (the only new things are the root's password, the connections made to and from the Wi-Fi interface and two static DHCP leases) :

root@OpenWrt:~# df -hT
Filesystem           Type            Size      Used Available Use% Mounted on
/dev/root            squashfs        2.3M      2.3M         0 100% /rom
tmpfs                tmpfs          13.6M      1.1M     12.5M   8% /tmp
/dev/mtdblock3       jffs2         320.0K    244.0K     76.0K  76% /overlay
overlayfs:/overlay   overlay       320.0K    244.0K     76.0K  76% /
tmpfs                tmpfs         512.0K         0    512.0K   0% /dev

As you can appretiate I only have 76kb of space left to use. I tried to install luci-proto-wireguard but I got:

I then tought I could install only the cli wireguard tool (wireguard-tools), but once again I hit the space problem, it ask me for at least 115kb when I barely managed to have 60kb space free after the opkg update.

So I had two ways to attack this problem:

Remove "unnecesary" packages (as ntp server/client, and all IPv6 related stuff). I tried it but somehow I removed a necessary package and I did left my device without internet access, so I needed to Factory Reset my router. To be completly honest I did not expect this work, as I'm trying to hit the pinata blindfold and with the 2 arms tied because my knowledge about this is not as big as I tought.
Install an older OpenWRT version, but I do not really if this is even possible

TL;DR
So after a really big introduction I think the real question to ask you guys is:

Is it possible to install yn a TP-Link TL-WR841N/ND v11 an older OpenWRT version that allows me:

Preferibly have web UI.
Allows me to connect to an internet Wi-Fi and in the same physical interface create an AccessPoint from which clients will have internet access, and of course share the internet with the LAN interfaces.
Have the necessary space to install the wireguard client (with web UI support if possible), and allows me to route all the traffic from my clients devices trough the vpn interface.
As security is not a concern for me I'm ok with the idea of use an older kernel in my router (current OS is using the 4.9.243 version.

?

Thanks for your help and you guys rocks!!! Your work is really amazing!!!

PS: I did not write the post with links because the automation tools of this forum deleted me my previous post

psherman · March 9, 2023, 5:08am

How did you attempt to remove the packages? Using opkg remove will actually end up consuming more space, and it will not free up any at all because the packages are 'baked-into' the firmware ROM and cannot be deleted.

Very unlikely to fit, but you can try.

I'm not sure if you're going to be able to fit this in, but you can try.

Security concerns typically only surface after your systems have been compromised and you look back wondering what you could have done differently. Don't underestimate the damage that can be done just by nabbing the password for your email, as an example.
But that notwithstanding...

You need to build your own custom image using the image builder. Follow this guide for what you can safely remove -- this needs to be done in the image builder 'recipe' to actually remove them from the base image.

You can take a look at my thread about doing a similar minimalist image, but in my case it was for a print server that does not have/route internet access (so I was able to remove -- or try to -- more than you can). You will need to keep the firewall and dhcp server and a few other things. Add wireguard into the image (it's more space efficient than adding it later because the image itself is compressed), and add a minimal LuCI to see if it will fit. You may have to spend a lot of time playing with the exact packages that will fit.

Oh, and one more thing -- you may run into issues down the road where settings get lost. That happens if the flash memory fills up and it can no longer write the config files. This will even be a risk if you change (not add) a config element such as the upstream wifi network credentials.

Another thing to note: I believe your device has only a single radio. If you attempt to run a wireless uplink (sta mode), that must be able to be established before the AP mode operation can start. If the upstream network is not available, the AP will never come up. There is a solution for this called Travelmate, but it's highly unlikely that you'll be able to fit that, too. So plan to always have a system with ethernet so that you can actually connect to the router if the wifi AP won't come up.

Yes, I am also using 18.06.9 on a 4/32 device for my print server, but it is on a trusted network and does not have access or exposure to the internet.

Cyborg · March 9, 2023, 5:24am

Hi psherman, and really thanks you for the very quick answer!

I removed the packages via the System > Software > Installed packages, which (and I'm just guessing here) indeed use the opkg remove command. And it is how you said: I was specting to free up space, but ended seeing less free space than before!!!

I really appretiate this comment.

I will defintly take a look into this!!!

Could I remove the dhcp server (letting just the dhcp client) and manually set the IP in the devices I will connect to the Wi-Fi?

Ohhh, I did see this when I restarted once, I thougth it was a common problem but did not knew the reason!!!

Thanks so much!!! You have send me a lot to diggest, I will read and then decide if it's worth spend the time (altougth the idea sounds really enjoyable I do not have much time free) in the trials and errors. Once again, thanks!!!

psherman · March 9, 2023, 5:31am

Yes, in theory.

A DHCP server isn't required, but it makes things much more complicated since you have to manually configure your connected devices. Removing it is fine if you don't mind the extra work to manually set static IP addresses on each device.
If you look at my linked thread, you will see that I was unable to remove the DHCP server (which I wanted to do for both space and functional reasons). It looks like I would have needed to compile the firmware from the source code, and it just wasn't worth the extra time and energy to do that in my situation.

Happy to help. I would highly suggest that you consider a newer device. There are some great travel routers are supported by OpenWrt and have far more space (both storage and RAM), and some of them are relatively inexpensive. Many new ones have 2 radios so you don't run into the "wifi won't come up" issue, and they have much better overall performance.

Cyborg · March 10, 2023, 4:40am

Hi @psherman... Here is my progress... Beforehand, sorry for the wall of text it comes. I give it a try to the Image Builder and here are my results:

Installing the necessary packages

I'm using an Ubuntu 20.04 VM in AWS, so following this guide I should install:

sudo apt install build-essential ccache ecj fastjar file g++ gawk \
gettext git java-propose-classpath libelf-dev libncurses5-dev \
libncursesw5-dev libssl-dev python python2.7-dev python3 unzip wget \
python3-distutils python3-setuptools python3-dev rsync subversion \
swig time xsltproc zlib1g-dev

Downloading Image Builder

You can download an archive that contains the Image Builder, it is usually located in the same download page where you find the firmware image for your device.
In my case should be the: https://downloads.openwrt.org/releases/18.06.9/targets/ar71xx/tiny/openwrt-imagebuilder-18.06.9-ar71xx-tiny.Linux-x86_64.tar.xz ? Am I right?

Profile

As there is no additional nor less packages for my device:

tl-wr841-v11:
    TP-LINK TL-WR841N/ND v11
    Packages:

I think I do not need to specify the PROFILE and let it use the default

Packages

I have some doubts here as in the doc it says to complete remove IPv6 support I need to just pass the CONFIG_IPV6=n option, but then in the little table showing me the packages that I can safely remove it says I need remove IPv6| -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables. So... should I pass the option CONFIG_IPV6=n AND in the PACKAGES delete (using the dash "-") OR should I just use the option CONFIG_IPV6=n?

PACKAGES="-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base luci-app-firewall luci-mod-admin-full luci-theme-bootstrap"

Luci minimal support

There is a think I didn't completely catch, as it's not explicitly said: when I compile using the Image Builder it doesn't come with Luci (web UI) support? I mean, in order to be able to use Luci minimal should I explicitly put the uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base luci-app-firewall luci-mod-admin-full luci-theme-bootstrap packages in the PACKAGES?

First dry run

In order to train myself and wait for your answers to my questions I will try:

make image PACKAGES="-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base luci-app-firewall luci-mod-admin-full luci-theme-bootstrap"

Once I run the previous command and see the output of the make end I proceed to inspect the folder

du -h bin/targets/ar71xx/tiny/* | grep -i wr841-v11
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-eu.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-us.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory.bin
3.6M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-sysupgrade.bin

Which rise some questions:

Why if I deleted some packages I do not see less occupied space by the firmwares I just created if I compare them with the "vanilla (original)" (3.8M)?
What version should I use the EU or the US? (I think I read somewhere that the EU version is limited to the output of the signal)
Should I use the PROFILE?

Test #2

Welp, the curiosity wins me, I did another test, but this time without specifying the Luci packages and with the CONFIG_IPV6=n:

make image CONFIG_IPV6=n PROFILE="tl-wr841-v11" PACKAGES="-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables"

Which gives me:

du -h bin/targets/ar71xx/tiny/* | grep -i wr841-v11
4.0K    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-device-tl-wr841-v11.manifest
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-eu.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-us.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory.bin
3.1M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-sysupgrade.bin

So I guess once installed the OS in the router I will see the real left space... But, without a Luci web, how should I proceed to configure my router? If I hook up an Ethernet cable to a client device the DHCP server will give my device an IP and I access the router via SSH using the .1 address?

Wireguard (?)

Question: As the main reason to do all of this is use Wireguard... What packages should I add in order to be able to use the Wireguard client with Luci support? luci-proto-wireguard?

Extra info

For what may server, my vanilla installation's packages:

root@OpenWrt:~# echo $(opkg list-installed | sed -e "s/\s.*$//")

base-files busybox dnsmasq dropbear firewall fstools fwtool hostapd-common ip6tables iptables iw iwinfo jshn jsonfigpio-button-hotplug kmod-ip6tables kmod-ipt-conntrack kmod-ipt-core kmod-ipt-nat kmod-lib-crc-ccitt kmod-mac80211 knat kmod-nf-reject kmod-nf-reject6 kmod-ppp kmod-pppoe kmod-pppox kmod-slhc libblobmsg-json libc libgcc libip4tc liihttp liblucihttp-lua libnl-tiny libpthread libubox libubus libubus-lua libuci libuclient libxtables logd lua luci  luci-mod-admin-full luci-proto-ipv6 luci-proto-ppp luci-theme-bootstrap mtd netifd odhcp6c odhcpd-ipv6only openwrtoot-envtools ubox ubus ubusd uci uclient-fetch uhttpd usign wireless-regdb wpad-mini

I have yet not installed anything on the router, I'm just making a little log/(self)explanation of my work here as I'm totally new to this

psherman · March 10, 2023, 5:34am

Take a look at the sysupgrade images -- those are varying in space (3.6MB and 3.1MB respectively).

You'd need kmod-wireguard and luci-app-wireguard.

I just tried doing the same thing and it will not fit...

Recipe:

make image PROFILE=tl-wr841-v11 PACKAGES="-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base luci-app-firewall luci-mod-admin-full luci-theme-bootstrap kmod-wireguard luci-app-wireguard"

Result:

[mktplinkfw] *** error: images are too big by 30482 bytes

Removing luci-app-wireguard allows it to complete successfully. If this is the 'killer app' you're going to have to figure out what else you can safely remove.

psherman · March 10, 2023, 5:38am

BTW, the reason to use profile in this situation is to reduce the number of images you're building. You go from buidling all the ar71xx-tiny devices to simply your own wr841. It's faster and takes up less unnecessary disk space on your computer/VM storage (not that a bunch of 4MB files really matters, but still, it does add up)

Cyborg · March 10, 2023, 5:43am

Ohhh, that's nice to ear!!! Thanks! I couldn't figure it out

Cyborg · March 10, 2023, 6:16am

Just to be clear: you are naming the sysupgrade images just for reference the difference in the varying space NOT because are those the one I will need to use to install on my router, am I right?

So... What if I choose to NOT install Luci at all? When I try the:

make image CONFIG_IPV6=n PROFILE="tl-wr841-v11" PACKAGES="-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables"

I can see in the make output:

1376256 bytes (1.4 MB, 1.3 MiB) copied, 0.00188107 s, 732 MB/s
Image Name:   MIPS OpenWrt Linux-4.9.243
Created:      Fri Mar 10 05:51:33 2023
Image Type:   MIPS Linux Kernel Image (lzma compressed)
Data Size:    1366920 Bytes = 1334.88 KiB = 1.30 MiB

So the question I think now is: without web configuration at all, how should I proceed in order to:

Connect to the router to configure it? I mean, I guess there is still the DHCP server running, so I could connect an Ethernet cable and plug it into my laptop, check in what subnet I got my IP address and then SSH into the router with what credentials??? If no Luci... can I brick my router?
Create an virtual interface that connects to a Wi-Fi, and use it as "WAN"... I think there is plenty of tutorials that teachs me how to do this in this very site.
Use the wireguard client to create a virtual interface and then connect to my server? I think I got my answer here, I even manage myself to find the documentation for openvpn client in OpenWRT... altought I think it weights (227KB) way more than the wireguard...
Create another virtual interface and configure it as Access Point using the radio interface? In where the DHCP server will lease IPs to the connected clients? This AP will be routed to the internet via the wireguard interface but of course will be able to reach local devices as well.
The LAN interfaces will still be reachable within the local network, but will reach the internet via the wireguard interface.
I was reading about Travelmate but it's available starting in the OpenWRT 20, so I think I will not be able to use it , do you know a about any other project that helps me with the problem?
Will my router's LEDs still be blinking in the way it's suposed to do?

psherman · March 10, 2023, 6:36am

If you have OpenWrt already running, you'll be using the sysupgrade version to upgrade to your new customized image. The "factory" image is only if you are installing from the vendor firmware or TFTP/recovery interfaces.

You don't need to completely omit LuCI -- you just can't fit WG + luci-app-wireguard

For initial config (and anytime you manage to get into that wifi bind I described), you'll use ethernet.
Connect using an ssh client. The credentials are: login = root, password = (blank for default install or whatever you set post-install).

LuCI or no-LuCI, you can soft-brick your router with bad configs, but you can recover using failsafe mode which will allow you to either fix the bad config, or reset to defaults.

All the 'virtual' interfaces are really just regular 'interfaces' from OpenWrt's perspective... no need to differentiate, and I'm not sure I'd even say they are 'virtual' anyway. Maybe just nit-picking or semantics, though.

Yes. This is a bit more effort, IMO, when working on the command line (vs web interface, if you decied to omit it). But totally doable. You just have to know what you're doing.

Yup... those guides will help you through the process.

OpenWrt is not recommended. It is extremely CPU intensive (and your system doesn't have a powerful CPU), and is much heavier in terms of storage and RAM utilization. I'd be surprised if you can get it to fit. It's also orders of magnitude more complex to setup.

Yup. Just tie an SSID to the lan network interface (which is actually already done by default).

Everyting on your lan will be on a common subnet, so no issue.

Actually, I think there might be an old version available for 18.06. The support thread begins in 2017.
But I think you won't be able to use it -- it's not going to fit. And I don't know of any other similar packages/projects. You could create your own super simple script to detect the condition and reset the wifi to a known good state.

If you want... you can define the LED behaviors in OpenWrt.

Cyborg · March 10, 2023, 7:19am

Holly molly, I didn't knew this! It all make sense now!!! I think you guys should add a little side note in the Image Builder documentation linking to these information, wherever it is.

By saying this, you mean that I can still install Luci but just not the wireguard's web configurator? And configure all related wireguard stuff via cli?

EDIT:
I tried this:

make image CONFIG_IPV6=n PROFILE="tl-wr841-v11" \
PACKAGES="-opkg -ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 \
-kmod-ip6tables uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base \
luci-app-firewall luci-mod-admin-full luci-theme-bootstrap \
kmod-wireguard wireguard-tools kmod-wireguard luci-app-wireguard"

Please, do note that I decided to NOT use the opk package as I think I will not install any more software... And you know what? It fits!!!

du -h bin/targets/ar71xx/tiny/* | grep -i wr841-v11
4.0K    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-device-tl-wr841-v11.manifest
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-eu.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory-us.bin
3.8M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-factory.bin
3.6M    bin/targets/ar71xx/tiny/openwrt-18.06.9-ar71xx-tiny-tl-wr841-v11-squashfs-sysupgrade.bin

My sysupgrade.bin it's only 3.6M!!! I will try it tomorrow morning (2AM here) and let you know how was it!

Thanks for the link!!!

Ohh, I came from the networking world, so I may be using the incorrect word here, sorry .But thanks for the note!!! I really appreciate it.

Yes, in the travel to my home from work today I tought about use a simple (b)ash script in order to set some if and some case and then do something... I still do not know pretty well what info should I need or what actions will I make, but that's an starting point: know that you know little.

@psherman once again... Thanks so much for your time!!! You have really guided me a lot in here!!!

Cyborg · March 10, 2023, 6:04pm

Welp....

I installed the cooked firmware into my router, and IT WORKS!!! I used this command line in order to make it:

make image CONFIG_IPV6=n PROFILE="tl-wr841-v11" PACKAGES="-opkg \
-ppp -ppp-mod-pppoe -ip6tables -odhcp6c -kmod-ipv6 -kmod-ip6tables \
uhttpd uhttpd-mod-ubus libiwinfo-lua luci-base luci-app-firewall \
luci-mod-admin-full luci-theme-bootstrap kmod-wireguard \
wireguard-tools kmod-wireguard luci-app-wireguard"

I guess I can also remove the luci-theme-bootstrap if I want to save even more space

I asked a friend of mine to share with me his Wireguard server with me to run some tests... I created the interface and...I'm connected!!!

But I loose the internet when I connected:

And of course it happens, I think I need to create a firewall rule in order to route all the traffic from the LAN and AccessPoint's connected devices to the wireguard recently created interface (please, do note that I did not use the "virtual", hahahha)

But I guess I will need some help here as I don't really understand the firewall Luci interface. When I create a new zone I don't really know what does means the Covered networks option:

So questions here are:

What networks should I add in the Covered networks option in order to route all the LAN interfaces and the AccessPoint clients to the wireguard interface?
Should I check the Masquerading option?
To and from what zones should I set the Inter-Zone Forwarding?

psherman · March 10, 2023, 6:41pm

Yes, but a minor point of clarification here... the firewall defines what is allowed but not what will actually be routed. The routing tables (often handled under the hood, but can be directly managed) will handle the actual routing itself. It's a subtle difference and usually can be ignored, but the firewall doesn't actually handle any of the routing... it just allows or denys the operation. Think of it like a a security checkpoint... the agents are not responsible for moving people from one area to another, they just make a decision about if that person is allowed through a gate or not.

OpenWrt uses a "Zone Based Firewall" (ZBF) approach. A zone may contain one more more networks, and those networks will be treated with the same zone-based rules (unless more granular rules are created, but that's a secondary thing). The 'covered networks' are the networks that are actually associated with the zone.

The input rule controls access to the router itself (i.e. administrative functions or any other services on the router itself) from any networks associated with the zone. The lan allows access by default, the wan zone does not.

Forward controls inter-network forwarding when the networks are part of the same zone. So if I create a second lan and put it into the lan zone where the default forward = accept, it means those two networks will be able to talk to each other. If I set it to reject or drop, it will block connectivity between the networks.

Output controls if traffic (that has entered and been processed by the firewall) is allowed to be output towards the desired zone/network. This is almost always accept for any normal network operatins.

Masquerading is used when you have a single upstream IPv4 address that need to be shared so that many devices (on a network behind a router) can gain access. Typically, the wan is masqueraded for this reason (most residential customers only get a single IP for all their devices). The Wireguard interface will also need to be masqueraded since it has a single IP that will be shared by devices behind the router.

These are the foward to/from zones... so lan can forward to wan by default (that's how the lan gets internet access), but wan cannot foward to lan (because we want to proect the lan from unsolicited connections from the internet/untrusted networks).

If you put the wg interface into the wan zone, it will be masqueraded already and your inter-zone forwarding is already configured.

If you put wg into a new firewall zone, you will need to allow lan > wg zone forwarding.

firefoxOnFire · May 7, 2023, 2:34am

Hey, can you share the data partition screenshot [ df -hT ] ? I want to know how much space you freed after custom install. I will try too.