Wanna see bandwidth broken down by wifi vs wired, per billing cycle

Fred will connect to internet via wifi. Sarah will connect by plugging her computer to the router via ethernet cable (wired). We need to measure bandwidth usage of wired vs wifi. We'd like to see a monthly log, where we set the beginning of the "month" to match the start of the billing cycle. How do we do this?

Sat Aug 5 17:01:24 PDT 2017 UDPATE: For anyone else interested in getting a solution, please feel free to skip to Wanna see bandwidth broken down by wifi vs wired, per billing cycle. Basically, I'll be trying luci-app-nlbwmon as my solution. :slight_smile:

vnstat is a simple, quite straightforward tool to log and accumulate interface traffic.

vnstat can monitor multiple interfaces separately, in your case wlanx and ethx

In /etc/vnstat.conf

# on which day should months change
MonthRotate 1

NB: By default, vnstat saves its database to a directory in /var, i.e. volatile ram. In order for the data to survive router reboots, either reconfigure it to write to flash/disk (keep in mind that the default saving interval of 30 minutes equals to ~17500 writes per year which isn't exactly healthy for flash memory) or use a backup script. The openwrt wiki page has a rather rudimentary backup script, I can provide a better one if needed.

2 Likes

Takimata, Thanks for your helpful reply.

If I use the backup script, where is the backup location?

And, yes, could you please share your script.

Thank you!

Strike that recommendation. The script on the OpenWrt wiki would save to a connected USB device ... if it worked. Which it doesn't, even in its rudimentary operation.

Let me clean up my script a little bit and get back to you.

1 Like

This is the script I wrote, goes into /etc/init.d/vnstat_backup

#!/bin/sh /etc/rc.common

EXTRA_COMMANDS="backup restore"
EXTRA_HELP=<<EOF
        backup  Backup vnstat database
        restore Restore vnstat database
EOF

START=98
STOP=10

vnstat_option() {
	sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" /etc/vnstat.conf
}

BACKUP_FILE=/etc/vnstat_backup.tar.gz
LOGGER_TAG=vnstat_backup
VNSTAT_DIR="$(vnstat_option DatabaseDir)"

backup_database() {
	if [ ! -d $VNSTAT_DIR ]; then
		logger -t $LOGGER_TAG -p err "cannot backup, data directory $VNSTAT_DIR does not exist (yet)"
	else
		logger -t $LOGGER_TAG -p info "backing up database"
		/bin/tar -zcf $BACKUP_FILE -C $VNSTAT_DIR .
	fi
}

restore_database() {
	if [ ! -f $BACKUP_FILE ]; then
		logger -t $LOGGER_TAG -p err "cannot restore, backup file does not exist (yet)"
	else
		logger -t $LOGGER_TAG -p info 'restoring database'
		[ ! -d $VNSTAT_DIR ] && mkdir $VNSTAT_DIR
		/bin/tar -xzf $BACKUP_FILE -C $VNSTAT_DIR
	fi
}

start() {
	restore_database
}

stop() {
	backup_database
}

backup() {
	backup_database
}

restore() {
	restore_database
}

Put this file into /etc/init.d as vnstat_backup, then

# chmod +x /etc/init.d/vnstat_backup
# /etc/init.d/vnstat_backup enable

and for good measure, create an initial backup:

# /etc/init.d/vnstat_backup backup

Aside from being able to called manually, it writes a backup of all databases into /etc/vnstat_backup.tar.gz when the router shuts down (reboot or shutdown), and restores it when the router starts up. You can create a cronjob if you want to backup the database in regular intervals (I do it daily).

However, I think this is only a stopgap. Ideally, this script's functionality could (and probably should) be integrated in some form with vnStat's init.d script, which could altogether use a little cleanup. (At the moment, for some reason, it allows restoring backups only for the currently monitored interfaces from a local file system and, curiously, from a web server, but doesn't create backups.) That reminds me that I should get in contact with @mkresin about that, as far as I can see he maintains the vnStat package for OpenWrt.

1 Like

Thank you.

Just to confirm, before a reboot or shutdown, the router will stay powered until the backup happens, yes? In other words, is it right to assume that the only time a backup will not happen is when the router loses power (e.g., unplugged, power outage)?

Gently, you still haven't answered a question I asked. :slight_smile: Does this script of yours back up to the flash memory? You mentioned that repeated saving to flash isn't good for the flash memory. Do we have alternatives? Could it save to the cloud (like my dropbox or my Google Drive)? Would saving to the cloud keep the flash memory from dying too soon?

Let me help @takimata to answer your question, yes as you can see the code above, it backs up the database to /etc/ you can upload the database to dropbox with https://github.com/andreafabrizi/Dropbox-Uploader. Don't worry flash memory doesn't wear out that easy but it's better not to write to it too frequently.

2 Likes

Thanks, hillz and takimata. I'll have to find the router from storage. Looking forward to getting this function working soon. :slight_smile:

Yes. Aside from blocking the shutdown until the backup is done, it only takes a tiny fraction of a second. vnStat databases are just a few kilobytes for each interface, additionally my script compresses all of them into a single tar.gz archive.

The problem I described above only arises if you have vnStat write to flash memory instead of RAM and don't change the default interval of 30 minutes, that would mean around 17500 writes a year, and that number of writes could cause problems in a foreseeable timeframe. However, a backup once a day and upon shutdown only makes up a few hundred writes to flash over the course of a year, and as @hillz says that should be fine.

I've updated the wiki with the script you provided https://wiki.openwrt.org/doc/howto/vnstat#method_2

Thanks. I slightly adjusted the wording in that section.

Thanks, English isn't my native language.

I'll take that as a compliment, but it's not mine either. :slight_smile:

Nope, I'm not the vnStat maintainer. Have a lool at https://github.com/openwrt/packages/blob/master/net/vnstat/Makefile to find the package maintainer.

But I have a backup solution for the vnstat database(s) as well, which copies the vnstat database once a day to the flash:

/usr/bin/vnstat_backup.sh

#!/bin/sh

. /lib/functions.sh

vnstat_option() {
        sed -ne "s/^[[:space:]]*$1[[:space:]]*['\"]\([^'\"]*\)['\"].*/\1/p" \
                /etc/vnstat.conf
}

local lib="$(vnstat_option DatabaseDir)"

backup() {
        local cfg="$1"
        local backup_dir

        backup_iface() {
                local ifn="$1"

                if cp -f "$lib/$ifn" "$backup_dir"; then
                        logger -t "vnstat" "Backup of database $ifn successfull"
                else
                        logger -t "vnstat" "Backup of database $ifn failed"
                fi
        }

        config_get backup_dir "$cfg" backup_dir

        [ -d "$backup_dir" ] || {
                echo "Error: backup directory not found" >&2
                exit 1
        }

        config_list_foreach "$cfg" interface backup_iface
}

[ -n "$lib" ] || {
        echo "Error: No DatabaseDir set in vnstat.conf" >&2
        exit 1
}

config_load vnstat
config_foreach backup vnstat

/etc/config/vnstat

config vnstat
        option backup_dir '/etc/vnstat/'
        list interface 'br-lan'

/etc/crontabs/root

34 01 * * * /usr/bin/vnstat_backup.sh
1 Like

Ah, it's @jow ... Alright, I should talk to him then.

I spotted your name because you were the one who committed restore functionality from local file system to the vnstat init.d script, obviously your backup script works in conjunction with that function.

Would you also agree that a backup functionality should go into the init.d script?

I am thinking about having some more parameters in /etc/config/vnstat (in addition to the already existing backup_dir, namely:

backup_dir            "/etc"
backup_on_shutdown    "0"
restore_on_startup    "0"
compress_backup       "0"

And the ability to call /etc/init.d/vnstat backup and /etc/init.d/vnstat restore Then it would all be in one central script, and integrated with UCI.

Also, very minor, the vnstat.conf supplied with the package still names the default interface to monitor as "eth0", vnstat.conf defaults to "br-lan". That could be synchronized. to the latter.

I am still extremely curious why and at which point the vnstat init script was extended to restore databases from a web server. That seems like a very specific use case to me. Unfortunately, it was inserted before the repositories were moved to Git, and that seems to have erased all change history. Or is there a mirror of the original svg change history somewhere?

No strong opinion on this. Backup is always something user/system specific and your solution most likely will not fit other peoples requirements. All I can say so far is that my little scripts does it job and that is enough for me.

The oldest reference I could find is https://dev.openwrt.org/changeset/11976/.

I realize that, being a programmer myself I am sensitive to that.

I disagree about "most likely not fit other people's requirements", though. The amendmends I propose above would cover all the backup strategies outlined in the OpenWrt wiki (backup on USB would just require to adjust the backup_dir option), and yes, yours, too. It should be a good baseline for what most people need, without the need for an additional script.

And of course, advanced users could still opt not to enable backup/restore through the init script (I feel they should be disabled by default anyway) and implement an own strategy.

Basically, it would be the exact backup counterpart to the already existing restore functionality already present in the init script, and it would not be enabled by default. So without any changes to the configuration the init script would do exactly what it does right now.

Ah, thanks, for some reason I could not spontaneously find the OpenWrt history anymore. So, interestingly that function was there from the beginning. Maybe @blogic can shed some light on it, if he still remembers it from 9 years ago.

@takimata,

How exactly do I implement your advice in Wanna see bandwidth broken down by wifi vs wired, per billing cycle ?

Can it be done in a GUI way in Luci? If it can only be done in CLI, could you kindly let me know how to do so?

Another question: I just came across https://wiki.openwrt.org/doc/howto/bwmon#view_historical_bandwidth_usage.
It says:
Approximate order from most-basic to most-feature-rich:
wrtbwmon < vnstat < YAMon < luci_app_statistics < bandwidthd < ntop

Below, it describes vnstat as "a console-based network traffic monitor". Am I wrong for wanting GUI? Are there advantages of a console-based traffic monitor that you don't get with a GUI traffic monitor?

For those wanting the gui version of vnstat why don't you make one yourself, here's the one that I made, I just redirect the output of vnstat to a log file and then output it on my local webpage

There is also luci-app-vnstat, which will create a number of plot representations of the vnstat data that can be looked at in the router's GUI... (on the CLI: "opkg update ; opkg install luci-app-vnstat")...
If you actually are willing to run the accumulation and display of the data on another host, have a look at:
http://richb-hanover.com/netflow-collectors-for-home-networks/
Which seems like a great solution if detailed analysis is desired...

Best Regards

1 Like