Monitoring bandwidth used per user. (Because metered internet.)


#42

Just a me too. I installed your LEDE master build from 20170814 on a wndr3700v2 and the IPv4 stats appear, but so far no values in the IPv6 fields (I will report back if IPv6 counters will start to show up after reboots).

Best Regards & many thanks for your great builds!


#43

Looks like the nlbwmon ipv6 data absence is a startup race-condition problem, or something like that.

I have ipq806x R7800 master r4696-df3295f50e and it has not shown IPv6 stats after the flashing yesterday. Only ipv4 stats have been visible.
Now I just restarted the nlbwmon service without doing anything else.
And nlbwmon started to count also ipv6 traffic...

I guess that the ipv6 ramp-up in the boot can take so long (odhcpd and odhcp6c) that nlbwmon starts before the ipv6 interface is properly up and misses the data.

Not sure about that, as I have not tried to look closer into source code, but that is my educated guess.


#44

Okay, restarting nlbwmon got me some IPv6 counter increases, but these do not match what I expect. I ran several speedtests (dslreports, either IPv4 or IPv6) and only with IPv4 I see counter increases that roughly match the tests transferred data volume, IPv6 seems to severely underestimate the volume... Not sure why.


#45

thanks @hnyman. Going to ipv6.google.com worked for me. So that means I'm using ipv6. Now I'm wondering whether the bandwidth stats collected by nlbwmon are incomplete because of nlbwmon's ipv6-collection issue.


#46

That should be easy to test, go to dslreports get a free registration and configure both an IPv4 and an IPv6 test, run both and see whether the nlbwmon counters increase (in the detailed information there is a view into textual output from the test that ends with reporting the actually transferred volume for both directions). In my quick test I just watched whether IPv6 tests increase the IPv6 counters (result the counters do not show all the IPv6 volume) and whether the IPv4 test increased the IPv4 counters (result: counters increased by the correct order of magnitude, do not expect the counter increases and the transferred volume to be identical, they are measured on different network layers after all). I did not test whether the IPv6 test increased the counters associated with my testing computer (I believe for your use case that is the relevant test, you want to correctly split a bill by usage and do not care about which protocol was used, correct?)).

Best Regards

P.S.: PLease note that the counter update happens every 30 seconds by default, so give at least 30 seconds after a speedtest for the couters to increase; also perform this test in an otherwise quite period.


#48

Moeller, Thanks so much for your helpful reply. :slight_smile:

I'm on a metered connection :blush:. How much bandwidth do the tests use?

This is bad, isn't it?

Not sure if correct order of magnitude is enough for me, is it? I need nlbwmon's counters to be accurate. Don't I need the counters to tell me -- for example -- that 492.5 MB of bandwidth was used when a 492.5 MB iso was downloaded?

Maybe I'm a noob for asking, but shouldn't that be my expectation? If nlbwmon's counter increases and the transferred volume aren't identical, then nlbwmon is still not doing the job I wish it did. Am I misunderstanding?

Oh, that's too bad. Wish you did. :wink:

Correct, I don't care whether the protocol I used is torrent or http or https or IRC or VoIP or ftp or whatever. I just need a way to measure my bandwidth use, just as it looks from the perspective of the ISP.

Hmmm... it will be quiet on the wifi. but the wired connection may be active. Will that be a problem? I'm guessing that this will be okay, right?


#49

You forget the protocol overhead. Packet reqs, acks, resends etc. For example, if you transfer a 100 MB file, you have typically transferred at least 100+3 MB download plus maybe 2 MB upload (packet requests, acks), making the down+up counters to show maybe 105 MB total traffic. If you experience packet loss (bad line, jam, QOS throttling), there may even be higher amount of re-transmissions.

I think that TCP/IP overhead (for TCP) is typically 2-4% in the optimal case.

And same limitation will hold true for vnstat, darkstat etc. They all measure the traffic from some network layer, but counter will not match the exact filesize of the transferred size.

But if you assume that each of these tools makes similar errors for all users, you can still use the traffic data for calculating the proportion that each has used internet, but not exact byte count.


#50

@hnyman, thank you for your reply. I misunderstood @moeller0 when he said that the counter and the volume are not identical. If all that he meant about differences was "overhead", as you so helpfully articulated, then I'm glad.

For (these) testing purposes, is there a way for me to see hourly or daily stats on nlbwmon?


#51

Okay, performing the proposed test myself, it seems that the IPv6 issue might be related to time-variant IPv6 addresses, at least my laptop does not use the IPv6 address anymore that nlbwmon believes it to use. Maybe the IPv6 address shown is purely cosmetic, but with IPv6 privacy extension IPv6 addresses will change for the same host frequently (and a host will have multiple of those addresses active at a given time). Whatever, currently IPv6 traffic does not seem to be reliably [quote="greenlaser, post:48, topic:2167"]
I'm on a metered connection :blush:. How much bandwidth do the tests use?
[/quote]

On a 50/10 Mbps link a test with 30 seconds run time (both for uplink and downlink) consumed:
84.83s Total megabytes consumed: 181.2 (down:149.9 up:31.4)
You can see the output of such a test under https://www.dslreports.com/speedtest/20410331

That, indeed is not what I expected.

Well, you will need to convince yourself, but I am only doing sanity checks (I have a non-metered link). Oh, no the counter should measure ethernet level volume, not payload volume, but that is fine your ISP will also charge you for all/most per-packet overhead, so the speedtest numbers will always just be lower limit that the real traffic will always exceed. The amount of the difference depends on both the per-packet overhead and the used packet size, but that is simply the reality in packetized data communication...

No, the thing is slightly more complicated. If you send a single full-sized packet (assuming ethernet) the MTU should be 1500, meaning the ethernet payload is 1500Bytes, but your ISP will still need to transfer the ethernet header as well, so
typically that would be:
4 Byte Frame Check Sequence (FCS) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) = 18 byte
So your ISP actually transferes 1518 bytes for you (and that is ignoring finer ethernet details like 7 Byte Preamble + 1 Byte start of frame delimiter (SFD) + 12 Byte inter frame gap (IFG) = 20 Byte)
So even on ethernet level the ISP sees 100-1001500/1518 = 1.19% more traffic than you would see by only looking at the ethernet payload.
But typically speedtests (and downloads) will only measure the TCP/IP Goodput and that is even smaller. For the following calculations I am making a few assumptions like no tcp options and no VLAN in use, but we still only look at one packet:
100-100
((1500 - 20 - 20) / (1518)) = 3.82%
So you ISP will see around 4% more bytes than you will measure, but since your ISP transfers these bytes on your behalf (and without those headers your data would go nowhere) the ISP probably expects you to pay for those as well...

nlbwmon ideally measures the 1518 bytes of this packet, but that is not fully clear to me. But best case really is tat nlbwmon measures something that is close to what your ISP is measuring, but do not expect bit-for-bit equality.

I did something better I used tcpdump to capture the test, and that speedtest fell back to IPv4, so the IPv4 couters increased as expected. Running a different IPv6 speedtest (confirmed by packet capture) resulted in traffic that was not measured at all. And yes that indicates that nlbwmon can be improved further.

Oh, I was talking about IPv4 or IPv6 here. Does you ISP actually enable IPv6?

That depends on the actual measurement you want to take, so I can not answer that.

Best Regards

P.S.: Maybe I should have read hnyman's response first :wink:


#52

This is good. If it's (only) 4% more, I'll just add 4% to my recorded bandwidth.

Please tell me if I understand you correctly: nlbwmon could be improved further in regards to measuring ipv6 traffic. But when it comes to ipv4 traffic, nlbwmon is measuring accurately, yes?

If nlbwmon is measuring ipv4 traffic/bandwidth correctly, then I'd be interested in seeing if I can force all my internet surfing/use to go through ipv4. Is there such a way? In other words, I'd like to block myself from using ipv6. How?

When I go to ipv6.google.com, I see the search page. So, yes, ipv6 is enabled.

I now just want to measure my use, which is wireless use (wlan0 and wlan1). I don't need to measure the bandwidth coarsing through the ethernet cable.


#53

Not so fast, the overhead is fixed per-paket, but the packet size is not fixed. If we look at a smaller packet size you will realize this point quickly (e.g. packet size 150 instead of 1500):
100-100*((150 - 20 - 20) / (168)) = 34.52%
really this is basic math, but it contains fractions... so it is obvious to everybody when looking at the formulas, but it is not intuitive. In short, you will not be able to correct for this by multiplying the transfer volume by a fixed factor. But if you take the number of packets transferred and figure out the part of the ioverhead your accounting method already handles for you you could correct things by addind NumberOfTranferrenPackets*NotAlreadyAccountedForOverheadSzePerPacket to the measured transfer volumes. I guess you get the idea...

Not what I tested, but foor IPv4 traffic the nlbwmon increases seemed correct by the order of magnitude, I would do closer rearexh if I would base finacial decisions on this (or simply agree with the party I want to cost share that we are fine with just using what ever measurement method to figure out the fraction each one needs to pay, so simply agree to accepts a little imprecision. But even then I would convince myself that I trust that I understand the measurements.).

But use of what wifi bandwidth or internet traffic?

Best Regards


#54

@moeller0,

After reading what you've written, with all that math, I'm feeling tired :laughing: This is trickier than I thought :slight_smile:.

Would using vnstat (on router) be an easier way to approximate my usage? I understand that vnstat monitors "LAN communication", but since I don't do any LAN communication (e.g. no peer-to-peer file sharing, LAN gaming), I don't mind paying the 100% of what vnstat calculates.

Please see comparisons of my vnstat and my nlbwmon here: Why such a huge discrepancy between vnstat and nlbwmon's bandwidth stats

The communication between my devices with the internet via wifi.


#55

@hnyman

nlbwmon is nice but I noticed that even though I did not include guest network in the settings, it was showing the traffic for it. In addition, it says lan: (no interfaces attached) although I have another access point and bunch of clients that are being monitored.

Is it possible to store the database on usb pen drive attached to the router, or external smb storage on another server? I am worried about wearing out internal flash storage.


#56

Yes, @metric! On http://192.168.1.1/cgi-bin/luci/admin/nlbw/config, click the Advanced Settings tab. The last blank is for "Database Directory". I have /mnt/sda1/ in the blank. That saves to my USB stick. Hope this helps, @metric. Your imperial friend. Lol.


#57

Is it possible to get luci-app-nlbwmon to look up the addresses of local hosts that are not on the local subnet/not handled by the on board dnsmasq server?

[In my network dnsmasq will know which other server to refer the lookup to]


#58

The reason nlbwmon doesn't work with IPv6 seems caused by missing ipv6 network in /etc/config/nlbwmon. I added below line in /etc/config/nlbwmon:

list local_network '::0/128'

And it worked for me:

xxxxxxxxxxxx… 7C:D1:C3:EB:3B:7F IPv4 8.75 MB 7.44 KP 739.39 KB 5.45 KP
IPv6 40.08 MB 30.17 KP 1.91 MB 18.46 KP
yyyyyyyyyyyy… DC:2B:2A:96:B5:7E IPv4 3.30 MB 2.66 KP 170.15 KB 2.15 KP
IPv6 4.05 KB 28 P 4.88 KB 36 P


#59

any idea how I can count traffic to/from direct to the router? I have transmission running on openwrt and would like to track that as well.


#60

Would it also be possible to have the database stored on a NAS ? if you mount it in OpenWRT ? or does it need to be locally stored?

Thanks,
Niklas


#61

Yeah, just mount the nas share (NFS or SAMBA) and link the directory to it.