Limit/Shape bandwidth for all on entire network/wifi

[Jannie007]

Good Day,
I know this question has been asked before but i am unable to get this to work.

I have a setup where my connection is metered. I pay per GB used but have high speeds.
The problem is that i share my wifi to family and friends. When they stream video the provider picks up that they have a high speed connection and streams hd/4k. This is a big problem for me and i dont see how normal bandwidth fairness is going to help me.

I would like to limit every connection/ip/mac to a defined speed (say 1mbs up and down)
Im running a Raspberry pi 3 B connected to a 4g modem via ethernet and broadcasting the wifi from the raspberry pi now

I have installed OpenWrt to try and sove this problem.
I have tried to setup Qos over Nftables, but im not getting it to work. When i do a speed test i still receive the full speed. Maybe someone can help met to set it up correctly?

I believe there should be simple way to do this over the entire network/wifi connection?

Any help would be much appreciated

[Jannie007]

Ok, so i seem to have done what i want using SQM and setting a limit on the wlan. Only thing that is bugging me is that my ping seems to be higher than usual?
Also ive read somewhere that using qos/sqm to do shaping on a metered connection is not a good idea as it does it by dropping packets? Is this so? If so then wont this method defeat the purpose as data will be "wasted" in order to shape the connection?

[moeller0]

@Jannie007

Also ive read somewhere that using qos/sqm to do shaping on a metered connection is not a good idea as it does it by dropping packets? Is this so? If so then wont this method defeat the purpose as data will be "wasted" in order to shape the connection?

As so often, the answer is it depends... Yes, the way TCP works is that is only slows down if it encounters missing packets (or explicit congestion marks vie ECN's CE-marking, if the TCP connection negotiates and honors ECN), so all traffic shapers will need to drop packets and/or CE-mak them. Interestingly it takes a relative small number of dropped packets to make a TCP-connection throttle itself, so you will typically not have that much dropped packets to worry about (assuming reasonably responsive TCP-flows, a single unresponsive flow can ruin your metering, but that is akin to a DOS attack and outside of the scope of post-bottleneck traffic shapers as ingress-SQM). Cake actually defaults to using ECN, so all you need to do is make sure all client computers negotiate ECN, but given your user base that probably is not a realistic option either...

[Jannie007]

Hi @moeller0,
Thank you for your time to reply, but to be honest, most of what you have said is going over my limited understanding of networks.
What is ECN?
Yes you are correct i dont have much control over my clients (other than to be rude and not to let them connect to the network).
The setup is almost akin to a free wifi hotspot in a airport for example only that it is fed by a per GB metered connection with high speeds (prepaid mobile broadband)
This setup is actually replicted on 3 sites and i would like to impliment the solution on all 3.

[moeller0]

@Jannie007

What is ECN?

See https://en.wikipedia.org/wiki/Explicit_Congestion_Notification for more details than you probably care in short ECN expands to explicit congestion notification as compared to the implicit congestion signaling from dropping packets. In the extreme, an overburdened router will have to resort to drop packages as that is the only way out of a prolonged congestion epoch that will avoid memory exhaustion and CPU overload on the router. But as long as the situation is not that dire ECN can help, both the overloaded router as ECN signaling is somewhat faster than the time endpoints need to believe a packet was dropped and the endpoints themselves, as ideally no packet needs to be dropped and resend (typically the TCP stack will wait for the missing packet before releasing and "later" data, so a dropped packet can be bad for latency sensitive applications).

@Jannie007

Yes you are correct i dont have much control over my clients (other than to be rude and not to let them connect to the network).

You could let your users know how to enable ECN in their devices (but the other end still needs to allow ECN as well, but Linux defaults to allowing ECN if requested by the other end for a long time now, so many internet servers are ready/willing to use ECN)...

@Jannie007

The setup is almost akin to a free wifi hotspot in a airport for example only that it is fed by a per GB metered connection with high speeds (prepaid mobile broadband)
This setup is actually replicted on 3 sites and i would like to impliment the solution on all 3.

Well, the good thing is that cake will use ECN for TCP connections that negotiated it and will resort to dropping for all others, but as I said typically not that much packets need to be dropped....

[Jannie007]

@moeller0

See https://en.wikipedia.org/wiki/Explicit_Congestion_Notification for more details than you probably care in short ECN expands to explicit congestion notification as compared to the implicit congestion signaling from dropping packets. In the extreme, an overburdened router will have to resort to drop packages as that is the only way out of a prolonged congestion epoch that will avoid memory exhaustion and CPU overload on the router. But as long as the situation is not that dire ECN can help, both the overloaded router as ECN signaling is somewhat faster than the time endpoints need to believe a packet was dropped and the endpoints themselves, as ideally no packet needs to be dropped and resend (typically the TCP stack will wait for the missing packet before releasing and "later" data, so a dropped packet can be bad for latency sensitive applications).

Thank you so much for the detailed explanation! I found the read very informative!

Ok so let me get this straight.
You are saying that cake will be able to perform what i want to implement and do it even better (more efficiently) because of ECN (will waste data less than any other method if the client supports it)?

Then that sound like the path i will def take.

For interest sake, do you know what the data usage/wastage is if one limits/throttles the bandwidth (of lets say a 25mbs down to 2mbs) Will it waste 23mbs while still logging the full 25mbs usage at the ISP?

If so then trying any implementation of shaping/throtteling/limiting of bandwidth that does not support ECN is a serious waste of effort and money (if you are on a metered connection)?

[moeller0]

@Jannie007

For interest sake, do you know what the data usage/wastage is if one limits/throttles the bandwidth (of lets say a 25mbs down to 2mbs) Will it waste 23mbs while still logging the full 25mbs usage at the ISP?

In theory with unresponsive DOS flows exactly that will happen, cake droppes all excess packets so only the configured shaper rate worth of packets are admitted into your own network, but TCP does not operate like that but will respond to a reliably detected missing packet by reducing its congestion window by a factor (0.5 for classic Reno, other TCPs use slighty different reduction factors), cake actually keeps statistics of the number of packets it drops, just run tc -s qdisc to get an idea how many packets were dropped:

qdisc cake 8056: dev ifb4pppoe-wan root refcnt 2 bandwidth 49Mbit diffserv3 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 34 mpu 68 
 Sent 152776401 bytes 151493 pkt (dropped 60, overlimits 187616 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 368Kb of 4Mb
 capacity estimate: 49Mbit
 min/max network layer size:           28 /    1492
 min/max overhead-adjusted size:       68 /    1526
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       3062Kbit       49Mbit    12250Kbit
  target         5.93ms          5ms          5ms
  interval        101ms        100ms        100ms
  pk_delay       11.5ms       4.63ms         50us
  av_delay       4.43ms       2.44ms         29us
  sp_delay         21us        605us         21us
  backlog            0b           0b           0b
  pkts             3143       144285         4125
  bytes         1730296    150317430       815833
  way_inds            0         1837            0
  way_miss          301         4383           32
  way_cols            0            0            0
  drops               1           59            0
  marks               0            2            0
  ack_drop            0            0            0
  sp_flows            0            1            0
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len          1492         1492          200
  quantum           300         1495          373

Here you see that on my 90Mbps link, shaped down to 49 Mbps over the course of 2.5 hours I only accumulated 1 + 59 = 60 dropped packets out of 3143 + 144285 + 4125 = 151553 transmitted packets. That said, the link was not really overloaded during that time. So under normal conditions the dropped packets should not be a big problem for your volume limited access.

[Jannie007]

@moeller0

Here you see that on my 90Mbps link, shaped down to 49 Mbps over the course of 2.5 hours I only accumulated 1 + 59 = 60 dropped packets out of 3143 + 144285 + 4125 = 151553 transmitted packets. That said, the link was not really overloaded during that time. So under normal conditions the dropped packets should not be a big problem for your volume limited access.

Thank you for that.

Can you please confirm, when i set shaping in SQM, does it shape the entire network to the speeds set (say 2mbs) or does each client have their own 2mbs connection when all are saturating their connections?
eg 2mbs devided among all or 2mbs for each client at all times while used together at max?

[moeller0]

@Jannie007

Can you please confirm, when i set shaping in SQM, does it shape the entire network to the speeds set (say 2mbs) or does each client have their own 2mbs connection when all are saturating their connections?

if you instantiate the shaper on your WAN link and configure both download and upload shaper rate, SQM will shape all traffic traversing that link.
SQM will by default do this in flow-fair mode, where all concurrently active flows get equal share of the total configured shaper rate. If you also want fair sharing between internal hosts (so that nobody can easily game the system by simply opening more flows) cake has you covered as well, just add:

        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost ingress'
        option eqdisc_opts 'nat dual-srchost ack-filter'

to your /etc/config/sqm... see the sing and dance section for details. Note that the ingress keyword will result in more aggressive shaping, as cake will try to shape the traffic such, that the incoming traffic actually traversing the download link does not exceed the set value (in the non-ingress mode cake will aim such that its own egressing traffic will meet the set rate), for your case, where you try to conserve the volume ingress should be the right choice. The ack-filter will reduce your upload traffic a bit and should help a bit in conserving volume as well. The nat keyword and the two dual-xxxhost keywords will instruct cake to first divide bandwidth fairly between all concurrently active hosts (it will still aim for flow-fairness within each host's traffic).

[Jannie007]

@moeller0

SQM will by default do this in flow-fair mode, where all concurrently active flows get equal share of the total configured shaper rate.

@moeller0

The nat keyword and the two dual-xxxhost keywords will instruct cake to first divide bandwidth fairly between all concurrently active hosts (it will still aim for flow-fairness within each host's traffic).

Im confused now.
This not what i need, is it? i do not want to split my 25mbs fairly between everone? And if i set SQM to 2mbs it will try to split the 2mbs fairly between everone, is that so?
I want to give each user a 2mbs pipe to use fully.
Do i understand something wrong?

[moeller0]

@Jannie007

This not what i need, is it? i do not want to split my 25mbs fairly between everone?

That is a policy question you need to answer, but if you shape the whole aggregate down to something smallish, sharing fairly between hosts seems reasonable to me.

@Jannie007

And if i set SQM to 2mbs it will try to split the 2mbs fairly between everone, is that so?

Yes, fairly between all concurrently active IP addresses, so a single active user will get up to 2 Mbps, two concurrently active users will get ~1 Mbps (or up to two Mbps if the other traffic subsides)...

@Jannie007

I want to give each user a 2mbs pipe to use fully.
Do i understand something wrong?

That is not something that SQM offers out of the box, you could create multiple SSIDs, one for each user and put an SQM instance set to 2/2 Mbps on each, but that is not terribly convenient...
I think that it should be possible to create something like that somehow (like creating a script to set-up a shaper tree for all IP addresses in your DHCP range all set to 2 Mbps) but not something I am versed in enough to even create pseudo code for...

[Jannie007]

@moeller0

That is not something that SQM offers out of the box, you could create multiple SSIDs, one for each user and put an SQM instance set to 2/2 Mbps on each, but that is not terribly convenient...
I think that it should be possible to create something like that somehow (like creating a script to set-up a shaper tree for all IP addresses in your DHCP range all set to 2 Mbps) but not something I am versed in enough to even create pseudo code for...

Thankyou for taking the time.
Multiple SSIDs are not going to be a solution in my case as i wont have any control over new users joining which SSID and how many users are on each SSID.

So i need somthing else other than cake/SQM.

It seems one of these might work:

There are lots of OpenWRT "Software" do the bandwidth per IP job. Google search those: eqos (tested working), luci-app-nft-qos (tested working, active development, MAC supported), qosv4(a tomato firmware shaping per IP merge, works on older version of WRT)
All those come with nice web GUI so don't need to bother setup of script / config files etc.

I have tried luci-app-nft-qos but i dont understand how to set it up to work and im unable to get eqos added to the setup.
Is there no other simple solution to this problem?

[Jannie007]

Ok, so i think ive relised that no form of QOS or SQM will work as they always want you to specify the total bandwidth (up and down) and then do their fairness calculations form there.

Being on mobile broadband my connection speed varies. One day it can be 25mb/s+ and the other its 8mb/s. i cannot see how QOS will help in such a situation where i want to conserive GB used because im on a metered connection.

I need a script/package that functions appart from the incoming bandwidth. It only needs to give each client a 1 or 2mbs line who joins the network.

Can anyone please direct me to where i can start to search in order to do this or is OpenWrt not able to perform this request?

Thank you

[Jannie007]

LIMIT BANDWIDTH per IP Network PFSENSE

LIMIT BANDWIDTH per IP Network PFSENSE

It seems as though this can be done using PFSense.
Can this be done using OpenWrt?

[trendy]

@Jannie007

Can anyone please direct me to where i can start to search in order to do this or is OpenWrt not able to perform this request?

Install luci-app-nft-qos for micromanagement of bandwidth per ip.

[dlakelan]

this is fairly advanced traffic control so there is no simple way, but it definitely can be done. the easiest way is probably to set up a large number of CBQ classes, and hash the flows into classes.

see manual for tc-cbq but seriously this will require some significant work.

[Jannie007]

Hi @trendy

Thank you for your reply.
I have tried to use it, but im unable to get it to work. It also is a QOS based approach which means i will have to supply it with my line speed, which is not constant. i dont think QOS/SQM fairness will work in this instance. Can you tell me what the difference between the static and dynamic options are? Perhaps the static option can get past this issue?
I have tried both but clients on the network are still able to make use of all the bandwidth. Im prob doing somthing wrong.
I need a script/package that functions appart from the incoming bandwidth. It needs to give each client a dedicated 1 or 2mbs line/pipe who joins the network.

@dlakelan thank you for your reply. Can you just confirm if im correct in saying that a QOS/SQM implementation will not work correctly in this instance when one wants to give each client a dedicated 1 or 2mbs line/pipe who joins the network irrespective of the ISP supplied bandwidth (which varies frequently and quite a bit) and one wants to conserve GBs used because its a metered connection?

Thanks for your advice on tc-cbq, ill have a look into it, but if you say that this request is fairly advance then i dont know if ill be able to implement it with my limited knowledge of networks.

Thank you again

[trendy]

I have not used it myself, so I won't be much more helpful here.
But not having a stable connection makes things quite difficult for every QoS solution I know.

[dlakelan]

SQM is not designed for your use case but QoS is a big tent and CBQ is part of that tent... so making a custom solution is possible. still I imagine you should have several years experience doing scripting and setting up say iptables or similar from scratch in order to feel comfortable with this project.

reading the Linux advanced routing and traffic control FAQ which is a decade or so out of date would still be a good starting point, but you should feel comfortable starting with out of date info and then reading up to date man pages and translating your knowledge etc.

[moeller0]

@Jannie007

I have tried to use it, but im unable to get it to work.

Mmmh, according to

SQM QoS limiting PS4 games console but nothing else

No devices were actually listed in nft-qos as you can see in the screenshot below. The global limits were enabled though to 50KBps which matches the limits I was getting on the PS4 of 5MBps (40Mbps). [Screenshot_20200717_132327]

All you might need to do in luci-app-nft-qos is to set a static default download and upload rates of 2000/8 = 250 KiloByte/second and things might just work?

EDIT: I used the wrong unit here initially: Kbps (Kilobit per second) instead of the correct and intended KBps (KiloByte per second) as @dlakelan thankfully pointed out. Thanks I corrected that above.