Why such a huge discrepancy between vnstat and nlbwmon's bandwidth stats

Let's look solely on the wlan0 (2.4 gHz) bandwidth, which is used by only one device, a Satellite laptop.

In nlbwmon's info, the Satellite's total bandwidth is less than 300 MB.
But in vnstat, wlan0's total bandwidth is 1.42 GB.

Can someone kindly help me understand the discrepancy?
Please know that both have the same billing cycle (same cutoff date).

For a bigger image of the above, click here.



For bigger image of above, click here.

Powered by LuCI lede-17.01 branch (git-17.152.82987-7f6fc16) / LEDE Reboot 17.01.2 r3435-65eec8bd5f

nlbwmon only counts traffic entering or leaving the local network, not the traffic within, while vnstat counts the total traffic on the interface even if it terminates within the network. Would that explain the discrepancy?

1 Like

Mmmh, I just ran a few test and nlbwmon seems to be leaky, in that it does not record all traffic. That might be related to both IPv6 privacy extension addresses (wild guess) and available CPU cycles (performing a flent test while running tcpdump on my lowly wndr3700v2 to capture the test traffic almost completely hid the roughly 1.3GB test volume from nlbwmon, but the router was fully pegged during the test (and the tcpdump load significantly reduced the trhoughput in the test, that wndr3700v2 really is a nice little router, but it is getting long in the tooth, at least wen doing silly things)).

Best Regards

@takimata, thanks for your reply. Yes, I think your answer helps to explain the discrepancy. But nlbwmon's stats still seem strangely low. In other words, I beileve I've used more bandwidth than nlbwmon is saying.

@moeller0, thanks for your reply. so you've given two additional explations.

  1. IPv6 (my nlbwmon doesn't seem to have as much info on ipv6 than ipv4)
  2. CPU cycles -- wow, for nlbwmon to almost completely ignore the 1.3 GB volume is an egregious mistake. Oh boy, I was so hopeful that nlbwmon would be the solution I've been seeking for six months. :crying_cat_face: :cry: :disappointed: I guess I'll have to keep searching.

All I want for Christmas is a reliable way to measure my bandwith usage, as seen in the perspective of the ISP (i.e. ignore local communications, etc.).

@moeller, might I ask you to please notify @jow. And maybe post a new issue on the nlbwmon github.

@greenlaser - sorry for piling on late but have you looked at YAMon (http://usage-monitoring.com)? I started working on YAMon 4+yrs ago because I wanted to validate the numbers in my ISP's bills. In YAMon, there's a feature that allows you to import data from your ISP so you can view the reports side-by-side.

Al

I am having the exact same issue. vnstats is capturing the actual traffic while nlbwmon isn't. I have seen that randomly and without any entry in the system log, nlbwmon stops monitoring. The process is still running in top, but it no longer uses CPU, it is idle forever. The only way to bring it back is to restart the process.
I can't test YAMon because I am using a router and AP without USB, so I am limited to the memory of the router, a Netgear R6020. So, I am now testing iptmon that uses the luci-app-statistics, and collectd packages so it can use the iptables of the firewall to track the amount of data.

However, the nlbwmon should be working. Any idea on why it stops monitoring?

Thanks

Yeah, I never figured it out. Given that we've correctly chosen Vnstat's interfaces so that we're not including the LAN (check), and accounting for differences in the way time period reporting works between the two (check), I have to conclude that either Nlbwmon has a bug that causes it to vastly underreport over time -- or the way in which it measures is simply flawed.

I should add that in my case I haven't seen Nlbwmon outright stop, which of course would be a problem. So, unless it's stopping for periods and then resuming, I don't think I have that particular explanation for the problem. If true, Nlbwmon apparently has multiple flaws.

One thing I do know is that Vnstat accurately represents reality, because I can compare it with what the ISP shows. They're very close.

Update:

I found this thread from several years ago. The author says:

"Vnstat looks at interface traffic counters which also factor in things like local broadcast traffic, protocol overhead etc. while nlbwmon solely counts connections which are destined from the local network to remote destinations."

So, unfortunately, Nlbwmon is not really the tool that most want, since reality is reality: there always is overhead, and it always counts. I'm not sure what he means by local broadcast traffic, since that shouldn't be coming into play unless someone has misconfigured Vnstat to count the LAN, which is not what we're talking about here.

So, when someone later in the thread poses this question:

"If I understand you correctly, nlbwmon will tell me how much bandwidth I've used, as seen from the point of view of the ISP. Is this correct?"

And the author answers "Yes, that is the basic idea."

...it's a little mystifying, since while that may be the basic idea, it's not at all what's happening. A correctly configured Vnstat is dramatically closer to what the ISP perceives, and that's readily provable.

Update 2: For those seeing nlbwmon errors in their syslog, that's a separate problem but one obviously that will make the discrepancy even worse. There are two main threads on some things to try to alleviate the errors.

vnstats and luci-statistics (collectd) provide more or less the same traffic. nlbwmon is always missing traffic, but it is a good tool to measure independent traffic.

There are other sidetools you can use such as iptmon package. it uses the firewall to track all the traffic going in and out and the source/destination device.