Node Exporter with uhttpd

Hi all

I have few OpentWRT routers at different places connected to the internet and I would like to monitor them at a central place. The monitoring solution I prefer right now is Prometheus in combination with the node exporter package (https://openwrt.org/packages/pkgdata/prometheus-node-exporter-lua). But I have not seen an easy way to use encrypted communication or some token based authentication in the standard node exporter if I would expose it to the internet (porpably it is just not designed to do that).
Therefore I like your help to find some way of accessing the router securely with authentication and limited permissions. I am able to think of the following approaches:

  1. modifying node exporter
  2. ssh tunnel
  3. some https web server approach
  4. vpn

Vpn would be my prefered way but sadly is not an option right now since I am on the wireguard hype train and the machine I like to use for running Prometheus is Ubuntu 18:04 without convenient way of installing kernel modules - "convenient" includes compiling from source (I have herd about a user space implementation.... does someone have experience with that?)
Ssh tunnel is implemented right now and works but is some hassle to set up non privileged user and keys. Overall I prefer something else.

What brings me to the point where I am stuck right now... The node exporter offers a lua script (/usr/bin/prometheus-node-exporter-lua) which prints what Prometheus needs to scrape. So it should not be to hard to teach a second utthpd process to listen on another port, execute that script, print back the output and secure that whole thing (ssl available by default)
But I am stuck with the that (of course testing only) setup:

config uhttpd 'prometheu'                                               
        list listen_https '0.0.0.0:65100'                               
        list listen_https '[::]:65100'                                  
        option home '/usr/bin/prometheus-node-exporter-lua'             
        option rfc1918_filter '1'                                       
        option max_requests '3'                                         
        option max_connections '100'                       
        option cert '/etc/uhttpd.crt'                      
        option key '/etc/uhttpd.key'                       
        option no_dirlists '1'                             
        option lua_handler '/usr/bin/lua'                  
        list lua_prefix '/metrics=/'                       
        option script_timeout '60'                         
        option network_timeout '30'                        
        option http_keepalive '20'                         
        option tcp_keepalive '1'  

curl --insecure https://localhost:65100/ prints the source code of that file and so does curl --insecure https://localhost:65100/metrics even though restarting the uhttpd gives me the following:

service uhttpd restart
Skipping invalid Lua prefix "/metrics=/"

I clearly have not comprehended how uhttpd or the lua integration works... sorry for that!
Either way I would greatly appreciate any hints to forwarding requests to the locally running node exporter server or executing that script by uhttpd as well es a completely different approach I maybe not getting right now. (maybe nginx would be a solution but if possible I'd stay "native OpenWRT" as long as possible)

Kind regards and thanks a lot!

How about OpenWisp?

@trendy I had a look at it right away and it seams to be very interesting approach. Sadly what I didn't tell you before is that I also intent to have some devices connected via wifi mesh which puts me in a situation where I already have come up with a configuration setup. I was more referring to "I would like to not mess with the standard configuration more then necessary".

Another think I like to add... Since from time to time connection drops I need to set low timeouts because otherwise the reconnecting client can not use the same port for request forwarding. (just more configuration think one could acoid)

But OpenWisp might be a solution for some else and I will consider it in the future :slight_smile:

Kind of answering my own question I now have successfully implemented a vpn connection via user space wireguard. To "briefly" summarize some things I have learned:
if Possible you of course want the original kernel implementation. If not I found the following 3 options on Ubuntu 18.04:

  1. wireguard-go
  2. boringtun
  3. snap package

1) is probably the most mature version of all and the one I am using right now. It needs to be compiled from source wich is a quite simple 3 step process (read up on github: https://github.com/WireGuard/wireguard-go). At least if you are able to realize that the documentation tells you that you need at least go version 13.1 which of course is not the one you get via apt install golang. Also read up on the WG_QUICK_USERSPACE_IMPLEMENTATION to make wg-quick (installed via ppa) use wireguard-go backend. With that it worked pretty well.

Of course this was the last one I tried and I went through to pain of failing the other two approaches.... the snap package is very easy to install and it also provide some wireguard-go implementation but it didn't work for me out of the box and I was not willing to dig into where the problem really was coming from inside of arbitrary /var/snap/directories. What I did figure out is that wireguard (when using wg for example) expects some socket at a specific point in /var. The snap version obviously has that one somewhere inside its personal sub directory.
Boringtun is pretty much wireguard-go written in Rust which I find pretty intriguing. As with the snap it was as simple as installing cargo via apt and then do cargo install boringtun. But who at thought.... I got a pretty alpha aka old version... That again could be considered my fault but well... it didn't work and I was unhappy about that.

Also yourISP may change your ip address from time to time which makes the other side of the wireguard connection still trying to contact your old address since it does not assume such changes. When you are sing a dns name for your router there is a script fixing this issue already. It is explained quite well in the Arch forum (https://wiki.archlinux.org/index.php/WireGuard#Endpoint_with_changing_IP)

Even tough that solves the problem at hand I am still wondering if there isn't any better way to expose a simple service in a secure manner.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.