Iptmon: simple iptables-based bandwidth monitoring

After trying most of the bandwidth monitoring solutions listed in the Wiki, I decided to roll my own using an approach most similar to wrtbwmon. I wanted the ability to monitor bandwidth per host both from within the LuCI interface (using luci-app-statistics) as well as to be able to export the same data to an external data store.

iptmon is a shell script intended to be triggered by DHCP that adds iptables rules to track RX/TX per host. Packet counts are then parsed by collectd using the iptables plugin (collectd-mod-iptables), which exports the data to an external InfluxDB instance, finally ending up on pretty dashboards served up by Grafana.

It's lightweight, cross-platform, doesn't require a constantly running daemon eating CPU and memory, and uses the already present packet accounting functionality of the Linux kernel.

Link to code: https://github.com/oofnikj/iptmon

Currently under active development and open to PRs :upside_down_face:

6 Likes

Update: works with IPv6 now.
Hosts are dynamically updated using the super-useful script-arp callback functions of dnsmasq.

Unfortunately LuCI doesn't display IPv6 firewall rules in the statistics view. I've submitted a PR to fix this.

1 Like

This looks really cool and I'd be interested in using this.

I am however slightly confused (and it's not your fault @oofnik).

It seems my dnsmasq automatically loads another custom dhcp-script out-of-the-box. To clarify, I don't have a dhcp-script line in my /etc/dnsmasq.conf file but instead, the custom script is automatically added in when I view /var/etc/dnsmasq.conf.

This is the custom script that my dnsmasq automatically executes:

#!/bin/sh

[ -f "$USER_DHCPSCRIPT" ] && . "$USER_DHCPSCRIPT" "$@"

case "$1" in
	add)
		export ACTION="add"
		export MACADDR="$2"
		export IPADDR="$3"
		export HOSTNAME="$4"
		exec /sbin/hotplug-call dhcp
	;;
	del)
		export ACTION="remove"
		export MACADDR="$2"
		export IPADDR="$3"
		export HOSTNAME="$4"
		exec /sbin/hotplug-call dhcp
	;;
	old)
		export ACTION="update"
		export MACADDR="$2"
		export IPADDR="$3"
		export HOSTNAME="$4"
		exec /sbin/hotplug-call dhcp
	;;
	arp-add)
		export ACTION="add"
		export MACADDR="$2"
		export IPADDR="$3"
		exec /sbin/hotplug-call neigh
	;;
	arp-del)
		export ACTION="remove"
		export MACADDR="$2"
		export IPADDR="$3"
		exec /sbin/hotplug-call neigh
	;;
	tftp)
		export ACTION="add"
		export TFTP_SIZE="$2"
		export TFTP_ADDR="$3"
		export TFTP_PATH="$4"
		exec /sbin/hotplug-call tftp
	;;
esac

Problem is, I don't actually understand what this script does and whether it's safe to overwrite with the script you're providing. Any ideas?

@thencein Thanks for your interest!
I actually didn't know about the hotplug subsystem while writing iptmon and only recently discovered it. It seems to be poorly documented unfortunately :frowning:

Looking at the code it seems like it would be a good idea to place wrappers in /etc/hotplug.d/dhcp and /etc/hotplug.d/neigh for iptmon instead of the current method of editing the dnsmasq base config directly.

For now the script in /var/lib/dnsmasq/dhcp-script.sh can be safely replaced by iptmon as long as there is nothing else that depends on scripts in either /etc/hotplug.d/dhcp/ or /etc/hotplug.d/neigh/.

All this script does is it calls the hotplug subsystem with the appropriate environment variables to trigger hotplug actions for DHCP and NDP/ARP events, respectively. This script will do nothing if there are no scripts in the aforementioned directories.

I'll do some testing and report back.


EDIT: There is a parameter available that's not listed in the Wiki called dhcpscript that can be used to override the default DHCP script triggered by dnsmasq. (source) I updated the Wiki :muscle:

So, instead of editing /etc/dnsmasq.conf it's probably 'cleaner' to add this to /etc/config/dhcp:

config dnsmasq
+       option dhcpscript '/usr/sbin/iptmon'

However we need --script-arp to be enabled in dnsmasq for iptmon, and there is currently no option for that, making it necessary to add this directly to /etc/dnsmasq.conf.
I've opened a PR for this: https://github.com/openwrt/openwrt/pull/2842

Just to update, I've taken your feedback and went ahead with replacing the 'default' script (saved in /usr/lib/dnsmasq/dhcp-script.sh) and replaced it with the iptmon script. After that, a simple chmod of the script followed by a restart of dnsmasq did the trick and I can now see the data flowing into influxdb nicely :smiley:

For now, it's going to be a waiting game to see whether anything has broken by me replacing the default script with your one. It could be that the hotplugs referenced in the old script are unused?

In the meantime, I can start visualising how I want my Grafana dashboard to look like :stuck_out_tongue:

Cool!

I actually updated the installation guide in a way that runs the default script in addition to iptmon.
Notice the line

[ -f "$USER_DHCPSCRIPT" ] && . "$USER_DHCPSCRIPT" "$@"

If option dhcpscript '...' is set, this doesn't replace the script, it just sets the variable USER_DHCPSCRIPT, which gets sourced by /var/lib/dhsnasq/dhcp-script.sh, preserving any previous functionality there may have been.

Feel free to use my dashboard for inspiration: https://grafana.com/grafana/dashboards/11858

And please, let me know if something is not working for you, I will be glad to look in to it.

I spent a couple of hours learning the build system, and finally I can announce that iptmon has been released as an installable package!
Version 0.0.1 (all architectures) now available on the releases page: https://github.com/oofnikj/iptmon/releases

Installation instructions have been updated for package installation.

2 Likes

Awesome! I tried out the ipk and it worked fine for me. :+1:

One question I have...let's say a guest device connects to my wifi - that device will now be 'collected' in the data collection.

Does iptmon automatically 'flush' out devices that are no longer present from data collection?

Otherwise, my Grafana dashboard will have lots of devices with 0 because they're guest devices from ages ago

Great, thank you for reporting.

iptmon will remove the iptables entries for the device once it becomes unavailable, so no new data will be collected, but historical data will be preserved both in the RRDs and in InfluxDB (if configured).

In Grafana, if there are no data points available in the selected time frame for a particular series, that series will not be displayed -- at least, that's how it's behaving for me in version 6.6.2.

Unfortunately it's not compatible with flow offloading
You can rework @jow 's nlbwmon

Yes, thanks, this was brought to my attention already. I will add a note in the readme about that.
I am familiar with nlbwmon, and at first I did try to build iptmon as a sort of 'extension' by running nlbw periodically and parsing the output - but decided against that approach for several reasons.

I am not really familiar with flow offloading (I'm working on x86 where it's not really necessary), but out of curiosity how does nlbwmon work with flow offloading? If I understand correctly, with this feature enabled, netfilter does not see the traffic at all. Maybe I'll need to do a little more reading...

nlbw uses netfilter conntrack accounting, and
this patch takes care of accounting on offloaded connections.

Interesting, thanks for sharing this link.
May need to pick up some embedded router hardware to test flow offloading.

You can just enable flow offloading on your x86, no need to get a router

I see why flow offloading it's missing in my config now after going through the LuCI code.

I'm using a non-OpenWrt kernel because I'm running in Docker:

This has already caused some unexpected results, as one might imagine. Adding to the list :sweat_smile:

Hello thank you for this very good plugin I am enjoying it's exactly what I was looking for, only thing that I would add is options for displaying packet graph and option to rename graphs :slight_smile: I am always confused which one is upload and download :slight_smile: I am looking at the moment for the code in system so I can disable it manualy but it will be great if all can do it by unchecking something :slight_smile: I am only interested in bandwidth in MB :slight_smile:

Hi @Liptovan, glad you found it useful.

I understand your confusion re: rx and tx, but I opted not to make it configurable because IMO the flow direction in all bandwidth monitoring tools should be indicated from the perspective of the router. This means that if you are watching a YouTube video on host my-laptop, for example, you will see this data flow in tx_my-laptop. I think that the option to reverse the direction would only increase confusion.

Re: packet graphs, these come by default in luci-app-statistics under /cgi-bin/luci/admin/statistics/graph/iptables/mangle-iptmon_tx / rx for transmit and receive. I didn't look in to disabling them, but I think it would require some code changes to this package.

All iptmon does is manage the appropriate iptables rules to be able to monitor and identify traffic; it can't decide what the actual monitoring tools show through the UI.

1 Like

Oh i didn't know it's complicated like that :slight_smile: I checked filesystem and I didn't found easy way to do it .. thx for information! :slight_smile:

Finally got around to adding support for static hosts.
Now any hosts defined in /etc/hosts or /tmp/hosts/* will be automatically included in the iptmon chains.

New release v0.1.0 ready for download.

1 Like

minor glitch @ master

dbg
+ eval '/usr/sbin/iptables -t mangle :iptmon_rx - [0:0]'
+ /usr/sbin/iptables -t mangle :iptmon_rx - '[0:0]'
Bad argument `:iptmon_rx'
Try `iptables -h' or 'iptables --help' for more information.
+ eval '/usr/sbin/iptables -t mangle -D FORWARD -j iptmon_rx'
+ /usr/sbin/iptables -t mangle -D FORWARD -j iptmon_rx
+ eval '/usr/sbin/iptables -t mangle -D iptmon_rx -s 127.0.0.1/32 -m comment --comment rx_localhost -j RETURN'
+ /usr/sbin/iptables -t mangle -D iptmon_rx -s 127.0.0.1/32 -m comment --comment rx_localhost -j RETURN
+ IFS=' 	
'
+ /usr/sbin/iptables -t mangle -A iptmon_rx -s -j RETURN -m comment --comment rx_
Bad argument `RETURN'
Try `iptables -h' or 'iptables --help' for more information.
+ printf 'added iptmon entry for %s %s\n'  
added iptmon entry for  
+ IFS= read -r host
+ eval dnsmasq_add ff:ff:ff:ff:ff:ff ::1 localhost
+ dnsmasq_add ff:ff:ff:ff:ff:ff ::1 localhost
+ mac=ff:ff:ff:ff:ff:ff