Migrating from Gargoyle - Bandwidth Management

Wondering if I can add additional subnets

The wording from the wiki appears to indicate I should be able to add something like:

	option subnets '10.10.1.0/24'	# Trusted
    option subnets '10.10.3.0/24'	# Guest
 	option subnets '10.10.10.0/24'	# IOT

and also tried as list

    option subnets '10.10.1.0/24, 10.10.3.0/24, 10.10.10.0/24'

but that doesn't appear to catch either.

Awesome! Let us know how the rest of it works for you.

I just have the one subnet. But I think I saw once that if more than one, they all goes on the one option line but the separator is odd. I see you tried comma. Maybe space or semicolon?

For more then one subnet, the filter may also have to change. Again, I have not tried it. Once you get it working, please post your config here. Thanks!

I'm working on an edit to the README that avoids the (dopey) mistake I made. I'll create a Pull Request when it's ready and you can review it. THANKS!

Thanks. I did try space colon dash splat (all failed).

I’ll pick my way through the different options and permutations. I understand the filter origin and syntax - good heads-up on that.

May be possible via procd init by setting separate instances for each separate subnets.

I’ll advise.

On my test device, I just tried a space as delimiter. It appears to have taken it. The service started and it generated files in /www/bandwidthd. It also gives me a subnet selection in index.html. However, I don't have anything generating traffic on a second subnet. So I can't do much more testing.

uci set bandwidthd.@bandwidthd[0].subnets='192.168.8.0/24 192.168.7.0/24'
uci commit bandwidthd
/etc/init.d/bandwidthd restart

If anyone is interested, I just added a sample hourly script that generates a report and limits bandwidth if over quota. You would have to modify it a great deal for your network and needs. But it is a place to start.

Once you have the bandwidthd subnets and filter settings figured out, you'll need to look at bandwidth_shape.sh. It assumes just one subnet. You'll have to create a new version. All commands that reference $BWSUBNET will likely have to become a for loop and perform same for each subnet. Best of luck!

I may have a simple solution for you. Since all your subnets are part of one larger subnet, just use 10.10.0.0/20. That way no modification to bandwidth_shape.sh should needed.

1 Like

I assume this doesn’t support groups of devices like Gargoyle?
Looking through the code it didn’t look too hard to achieve given the sql queries.

I also used groups in Gargoyle. The --include and --exclude should allow you to do something like Gargoyle groups. I tried to demonstrate that in sample_report_hourly.sh. I recently added that to the Github. See what you think.

Thank you for the question. I should make it clear that this post is really about limiting access to the Internet. It's not about bufferbloat or fairness. I have Mediacom. They have a data cap per month that is expensive to exceed. The primary devices consuming my Internet bandwidth are smart TVs and streaming via Chromecasts. Most of these services don't have a built in method to limit data usage. So, I want to automate the process of tracking the device's data usage and slowing down their data consumption when it gets out of hand.

2 Likes

Not quite a month tracking this process and subnets appear to be working - space delimited in the config.

This is the filter I came up with that appears to accurately capture all subnets 10.10.1.0/24, 10,10.3.0/24, and 10.10.10.0/24

option filter 'ip and ether host 60:xx:xx:xx:xx:AF and not host 10.10.1.1 or ether host 00:25:xx:xx:xx:99 and not host 10.10.3.1 and not host 10.10.1.1 or ether host 02:25:xx:xx:xx:99 and not host 10.10.10.1 and not host 10.10.1.1

I've also taken the liberty to co-opt your bandwidth_used.sh script for a somewhat different use. I wanted to see total bandwidth over a span that showed each ip as a percentage of the span total.

1 Like

Dear RVgo4it
Thanks a lot for your material.
I recently installed OpenWrt on my Netgear R6800 with the goal to setup qouta/access control:

OpenWrt 22.03.2, r19803-9a599fee93

Some 27d ago RuralRoots was so kind to refer me to your project.

Bandwidthd OpenWrt package works as expected and graphs are displayed properly.
:slight_smile:

But coming to the bandwidth_used.sh my instance runs into trouble:

root@OpenWrt:~# /mnt/sda1/bandwidthd/bandwidth_used.sh
Bandwidth Quota Daily Report for: 2022-12-02 08:36
Time span (from - to) Total Bytes % of 1073741824 Kbytes
Error relocating /usr/bin/sqlite3: sqlite3_error_offset: symbol not found
Error relocating /usr/bin/sqlite3: sqlite3_column_origin_name: symbol not found
Error relocating /usr/bin/sqlite3: sqlite3_column_database_name: symbol not found
Error relocating /usr/bin/sqlite3: sqlite3_column_table_name: symbol not found
2022-12-01 12:00 - 2022-12-02 12:00 0 Kbytes 0%
[10x...]

RuralRoots patiently -thank you so much :sunglasses:- accompanied me to locate and solve the error, but finally we realized that in this forum we will find the solution.

Error relocating /usr/bin/sqlite3: sqlite3_column_database_name: symbol not found

it looks like a

Typically build relocation/linker issue, but I am out of my league here.

I still don't want to give up and go back to the OEM firmware-do you see the source of the problem of my instence?
Any help welcome :slight_smile:

Stargate

That is odd. I have not run into that. I have the quota system running on versions OpenWrt 21.02.3 r16554-1d4dea6d4f and OpenWrt 22.03.2 r19803-9a599fee93, both mvebu/cortexa9 and x86/64 hardware. I took all the latest updates for sqlite3 and it still worked fine.

Those relocating errors look like function names for sqlite lib, likely from libsqlite3-0 package. I would try removing all the sqlite packages, including bandwidthd-sqlite, and reinstall them.

Try this simple command just to see if sqlite3 cli is working and the database is good.

sqlite3 `uci get bandwidthd.@bandwidthd[0].sqlite_filename` 'SELECT sum(total) as total FROM bd_rx_total_log'

It may be something as simple as a corrupt db. Since the db is not needed for the charts, they can look fine even if the sqlite db is sick. To make bandwidthd build a new db, try these commands:

/etc/init.d/bandwidthd stop
rm `uci get bandwidthd.@bandwidthd[0].sqlite_filename`
/etc/init.d/bandwidthd start
# wait a few minutes...
sleep 300

Then try the simple command again just to see if sqlite3 cli is working and the database is good.

RVgo4it

PS: Just about to go on another RV trip and will have very limited network access for over a week. Good luck!

1 Like

Perfect - these recommendations made it :slight_smile:

sqlite3 uci get bandwidthd.@bandwidthd[0].sqlite_filename 'SELECT sum(total) as total FROM bd_rx_total_log'

produced similar error-messages.

/etc/init.d/bandwidthd stop
rm uci get bandwidthd.@bandwidthd[0].sqlite_filename
/etc/init.d/bandwidthd start
# wait a few minutes...
sleep 300

did not help to eliminate the error-messages.

before ------------------------------->

after --------------------------------->

and bandwidth_used.sh works
libsqlite3-0 is now on version 3400000-1

Thanks a lot!

I have been playing with this off and on trying different libpcap filters, reports based on different subnets, other scenarios, . . . and started seeing anomalous behaviour specifically with accurate capture on the second and third subnets.

I caught this when I moved my IOT network off the bridge and pretty well all traffic on that subnet stopped.

I started sniffing through the source tree and came upon this little gem; bandwidthd -l. bandwidthd has a Pseudo-device that captures traffic on all interfaces option dev 'any' and once configured and option filter removed everything started flowing accurately including gateway and broadcast ip's.

added `option filter 'ip and not host 10.10.1.1 and not host 10.10.3.1 and not host 10.10.10.1 and not host 10.10.1.255 and not host 10.10.3.255 and not host 10.10.10.255' all is good.

I tried the 'any' option once too. But I was still using the bridge at that time, so lots of duplicate traffic was counted. Sounds like a great solution for you since you are not using the bridge. Awesome!

What an amazing Post @RVgo4it .

tried it on 22.03 and it works great.
but I have 2 questions

  1. for folks with x86 builds like me is there any way to increase the data scrape frequency from bandwidthd? I do not care about the graphing itself as much as I care about the actual data. upon inspecting the sql file it appears that the data is collected every 200 seconds?! would be great if I could collect it every 60 seconds or so.

  2. excuse me if it seems a bit silly (my Shell coding is a bit rusty) but I wonder if the bandwidth_used.sh can be modified to take a date range instead of setting a specific Turnover. I might try to fork it and see how it goes also creating a package for openwrt would be a fun little project for me

Thanks for your comments.

  1. I checked what little documentation I have on the interval. It appears fixed at 200.
  2. A date range would be a very simple script. Feel free to reuse my SQL statements. Everything else from bandwidth_used.sh will likely not be needed.
1 Like