Router with efficient bandwith limitation

Simbud · January 5, 2024, 9:25am

Hello all.

thanks for all people responding and helping others

After spending few hours of googling and reading posts on this forum, I must admit I'm still lost.

I have a quite low internet ADSL connection : 16 Mb/s download, 900 Kb/s upload.
My family has 3 computers and I own a small Gli.net Wifi router with OpenWRT I use for SQM that is efficient for QoS for HTTP, downloading, video streaming, etc.
But when come games updates, SQM is useless and I encounter bufferbloat issues, as I think SQM is lost with P2P used by Epic, Steam, Blizzard, etc. for their updates.
I know some have software bandwidth limitation, but some doesn't.

So I'm looking for a real bandwith limitation per device/MAC/port that won't be fooled by P2P / games updates.

I've thought about TL-WR940N but the OpenWRT warning about it's RAM made me unconsider this choice.

I've seen package nft-qos but not sure it would fix my problem, and seems not to be maintained anymore : https://github.com/openwrt/packages/tree/master/net/nft-qos/files

My budget is about 80$ for a used device (so let's say it would cost approx 150/200$ new)

Thanks for any help / clue

eduperez · January 5, 2024, 9:33am

Are you sure you need a new device? 16Mbps should not be very demanding... I would start by deciding what software you need, and try it in your current device.

Simbud · January 5, 2024, 9:41am

Thanks for your reply, but I don't understand what you mean by "deciding what software you need, and try it in your current device"

I don't need any software, just bandwidth limitation. Or maybe you're talking about a software on PC limitating bandwidth ?

eduperez · January 5, 2024, 9:47am

I am referring to the software (or "package") you are going to install on the router to do the bandwidth limitation.

Simbud · January 5, 2024, 10:31am

well... that's part of my question, event if I wasn't so clear.

Can OpenWRT do it efficiently on any decent router ? With qos-nft ? Other package ?
Or must I buy a router and keep it's bandwith limitation software ?

flygarn12 · January 5, 2024, 11:01am

What bits are you going divide between the clients?

Netflix and any other streaming service pretty much assumes at least 9Mbps to get somewhat flowing HD quality. If we talk 4K or Dolby video and or sound, well that will never happen on this internet line since we then talk about 20-40Mbps download speed.

So whatever you run will by itself need all or nothing of what you have.

Can the game run on this as single client and with a clean router? If that isn’t the case QoS won’t matter since that by it self is a data management service so it adds process time.

Lynx · January 5, 2024, 11:25am

To me this just suggests cake or whatever qdisc has not been setup correctly. The requisite throttling should be effective to prevent bufferbloat regardless of the type of traffic.

Simbud · January 5, 2024, 7:16pm

Thanks for the clues guys.

I wanted to check my SQM config, but it's broken. I'll try to reinstall it and/or reinstall OpenWRT.

I'll also try tips in this topic : https://forum.openwrt.org/t/possible-sqm-issue-p2p-download-bogs-down-the-network/124356/6

For information, I'm using this :

moeller0 · January 6, 2024, 9:41am

Please log into your router via ssh and run the following commands then copy and paste the output:

cat /etc/config/sqm
ifstatus wan | grep device
tc -s qdisc

pythonic · January 6, 2024, 11:12am

I've not tried to use it this way myself but you may wish to have a look at Gargoyle which is built on OpenWrt (most recent release built from 22.03; your GL-AR150 is supported) with it's own bandwidth management tooling. If you're already running native OpenWrt (not gl.Inet's derivative) you can install it via OpenWrt's UI using the sysupgrade image (but don't save configuration!). If it doesn't work for you, you can then convert back to OpenWrt the same way. Please ask any questions about Gargoyle in its forums though, not here.

moeller0 · January 6, 2024, 12:01pm

How do you measure that? Are you trying to play a game on the same computer while updating games, or do you notice the effect on other machines?

Are these sync speeds or are these results from sppedtests?

Maybe you can run a test under

and post a screenshot of the result here? Please note the result contain a map with a rough location of your router (as precise as geoIP allows) and your current IP address, depending on your personal privacy preferences you might want to redact these pices of information, but please tell us your country and ISP.

Simbud · January 7, 2024, 4:01pm

Thanks again for your answers and clues

The "TypeError" I got on many pages on Luci UI were du to Brave browser : none problem under Firefox ...

I've upgraded to the latest versions, now 23.05.2

To pythonic : thanks, I'll surely give a try

To answer moeller0 :

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '14000'
        option upload '700'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option linklayer 'none'
        option debug_logging '0'
        option verbosity '5'
        option squash_dscp '1'
        option squash_ingress '1'

root@OpenWrt:~# ifstatus wan | grep device
        "l3_device": "eth1",
        "device": "eth1",

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 66004216817 bytes 44964158 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8039: dev eth1 root refcnt 2 bandwidth 700Kbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 1735911969 bytes 24949710 pkt (dropped 32824, overlimits 11121141 requeues 0)
 backlog 846b 13p requeues 0
 memory used: 1148Kb of 4Mb
 capacity estimate: 700Kbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        700Kbit
  target           26ms
  interval        121ms
  pk_delay       92.7ms
  av_delay       7.27ms
  sp_delay        4.9ms
  backlog          846b
  pkts         24982547
  bytes      1747508053
  way_inds       366608
  way_miss        30242
  way_cols            0
  drops           32824
  marks               0
  ack_drop            0
  sp_flows           10
  bk_flows            2
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 65049998570 bytes 44630980 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 803a: dev ifb4eth1 root refcnt 2 bandwidth 14Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 63586684038 bytes 43215474 pkt (dropped 1415492, overlimits 86142692 requeues 0)
 backlog 20872b 14p requeues 0
 memory used: 398Kb of 4Mb
 capacity estimate: 14Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         14Mbit
  target            5ms
  interval        100ms
  pk_delay       13.6ms
  av_delay       6.08ms
  sp_delay        815us
  backlog        20872b
  pkts         44630980
  bytes     65674832290
  way_inds       235653
  way_miss        35809
  way_cols            0
  drops         1415492
  marks               1
  ack_drop            0
  sp_flows            6
  bk_flows            5
  un_flows            0
  max_len          1514

Here are my settings in UI :

Here's how PC are using internet:
Diagram1

I've measured my speed test with speedtest.net, and that's coherent with what is displayed on my modem.

The problem is when there's a game update on PC 3 (and that happens often), internet is unusable and/or very laggy on other PC, especially with movie watching from SSH fuse mount (I know, that's not a very common practice, but works well when no games updtes )

I've tried some intensive download (with speedtest) simultaneously on each 3 PC : bandwidth is well split between all of them, approx 5 Mb/s on each. And when I stop the download on 1 of the PC and try to play : it's ok => others PC get approx 7 Mb/s and I can play. But downloading/watching from the ssh mount is very laggy/stopped.

This is my 2 years experience of this setup with SQM that makes me think SQM can't handle P2P/games update well. No proof, just empirical experience.

moeller0 · January 7, 2024, 5:47pm

Well, at 16 Mbps (and 0.9 Mbps in the other direction) things can get tight.
However, I think your confoguration is not as tricked out as it can be, please try to replace your /etc/config/sqm with this:

config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '14000'
        option upload '700'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option linklayer 'none'
        option debug_logging '0'
        option verbosity '5'
        option squash_dscp '1'
        option squash_ingress '1'
	option itarget 'auto'
	option etarget 'auto'
	option squash_dscp '0'
	option squash_ingress '0'
	option qdisc_really_really_advanced '1'
	option eqdisc_opts 'nat dual-srchost memlimit 32mb'
	option iqdisc_opts 'nat dual-dsthost ingress memlimit 32mb'
	option linklayer 'atm'
	option linklayer_advanced '1'
	option tcMTU '2047'
	option tcTSIZE '128'
	option tcMPU '88'
	option linklayer_adaptation_mechanism 'default'
	option overhead '44'

And please do not deselect the advanced option checkboxes in the GUI...
Somehow the values shown in the GUI and in your /etc/config/sqm seem out of sync...

What all of this will do is make the per-packet overhead accounting more precise, it will also do the correct accounting for ATM/AAL5 cell quantization, and it also do a stronger form of per-internal-IP fairness...

Also please do the following:

disable sqm: /etc/init.d/sqm stop
tc -s qdisc and copy and paste the output to this thread
run a test under https://speed.cloudflare.com and post a screenshot of the results here in this thread (the speedtest.net link above does not show your test result)
enable sqm again: /etc/init.d/sqm start
tc -s qdisc and copy and paste the output to this thread
immediately after the test and even before taking the screen shot tc -s qdisc and copy and paste the output to this thread

The goal is to figure out how high we can set the upload shaper safely, at that low rate every Kbps more can help. And how well sqm works with a well behaved load...

Yes, by default cake uses per-flow fairness and that works mostly well unless one application uses cobsiderably more active flows than the others, but that is pretty much what manny P2P applications do. Now cake offers some better isolation modes to counter that (triple-isolate, dual-srchost/dual-dsthost) but these work best if they can peek into the NAT connection database...

Anyway, please consider running through the list of changes and measurements as it might help improve your situation a bit... at 14/0.7 your link will always leave something to be desired, but I hope we can help make things a bit better.

Simbud · January 7, 2024, 10:27pm

Thanks for clear procedure.

I've ticked the checkbox "advanced options" in GUI for SQM.
By default, it's unchecked, and this procedure tells to let it unchecked

And for the others settings :

root@OpenWrt:/etc/config# cat /etc/config/sqm

config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '14000'
        option upload '700'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option linklayer 'atm'
        option debug_logging '0'
        option verbosity '5'
        option squash_dscp '1'
        option squash_ingress '1'
        option overhead '44'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '0'
        option itarget 'auto'
        option etarget 'auto'
        option qdisc_really_really_advanced '1'
        option eqdisc_opts 'nat dual-srchost memlimit 32mb'
        option iqdisc_opts 'nat dual-dsthost ingress memlimit 32mb'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '88'
        option linklayer_adaptation_mechanism 'default'

Test :

root@OpenWrt:/etc/config# /etc/init.d/sqm stop
SQM: Stopping SQM on eth1

root@OpenWrt:/etc/config# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 80302412490 bytes 55690656 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 8514 bytes 90 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
root@OpenWrt:/etc/config#

Cloudflare test with no traffic on any pc :

Five seconds after the test :

root@OpenWrt:/etc/config# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 80475364756 bytes 55831592 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 10236858 bytes 41139 pkt (dropped 0, overlimits 0 requeues 3)
 backlog 0b 0p requeues 3
  maxpacket 1454 drop_overlimit 0 new_flow_count 5 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

Then:

root@OpenWrt:/etc/config#  /etc/init.d/sqm start
SQM: Starting SQM script: piece_of_cake.qos on eth1, in: 14000 Kbps, out: 700 Kbps
SQM: piece_of_cake.qos was started on eth1 successfully
root@OpenWrt:/etc/config#
root@OpenWrt:/etc/config# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 80478264792 bytes 55836271 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8051: dev eth1 root refcnt 2 bandwidth 700Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 mpu 88 memlimit 32Mb
 Sent 21524 bytes 174 pkt (dropped 0, overlimits 65 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 8Kb of 32Mb
 capacity estimate: 700Kbit
 min/max network layer size:           40 /    1278
 min/max overhead-adjusted size:      106 /    1484
 average network hdr offset:            7

                  Tin 0
  thresh        700Kbit
  target           26ms
  interval        121ms
  pk_delay       15.4ms
  av_delay        617us
  sp_delay         33us
  backlog            0b
  pkts              174
  bytes           21524
  way_inds            0
  way_miss           26
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1292
  quantum           300

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 114397 bytes 200 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8052: dev ifb4eth1 root refcnt 2 bandwidth 14Mbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms atm overhead 44 mpu 88 memlimit 32Mb
 Sent 117197 bytes 200 pkt (dropped 0, overlimits 85 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 6Kb of 32Mb
 capacity estimate: 14Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:            8

                  Tin 0
  thresh         14Mbit
  target            5ms
  interval        100ms
  pk_delay        938us
  av_delay         88us
  sp_delay         39us
  backlog            0b
  pkts              200
  bytes          117197
  way_inds            0
  way_miss           26
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           427

Note: I've set bandwidth in OpenWRT/SQM at approx 90% of my real bandwidth, as advised in this guide : https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm.

moeller0 · January 8, 2024, 11:15am

You have these options twice, please delete the first pair...

This was without SQM, now I want to see a test with SQM (and the tc -s qdisc output from just before and just after)...

Sure, but these are only starting points and we can actually do better than that! But let's first see whether this configuration actually helps with your initial problem?

Simbud · January 8, 2024, 1:10pm

Understood.

Thanks for your time and effort.

I'll do tests tonight. Have a good day.

Simbud · January 8, 2024, 6:54pm

I've fixed the /etc/config/sqm file
Then I've rebooted the router.
Then :

root@OpenWrt:~# /etc/init.d/sqm status
active with no instances
root@OpenWrt:~#
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 1014791 bytes 1867 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 800d: dev eth1 root refcnt 2 bandwidth 700Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 memlimit 32Mb
 Sent 83289 bytes 727 pkt (dropped 0, overlimits 355 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 14Kb of 32Mb
 capacity estimate: 700Kbit
 min/max network layer size:           28 /    1500
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:           13

                  Tin 0
  thresh        700Kbit
  target           26ms
  interval        121ms
  pk_delay       8.82ms
  av_delay        781us
  sp_delay         32us
  backlog            0b
  pkts              727
  bytes           83289
  way_inds            0
  way_miss           68
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 494474 bytes 837 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 800e: dev ifb4eth1 root refcnt 2 bandwidth 14Mbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms atm overhead 44 memlimit 32Mb
 Sent 506192 bytes 837 pkt (dropped 0, overlimits 346 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 12Kb of 32Mb
 capacity estimate: 14Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:           13

                  Tin 0
  thresh         14Mbit
  target            5ms
  interval        100ms
  pk_delay        665us
  av_delay        125us
  sp_delay         34us
  backlog            0b
  pkts              837
  bytes          506192
  way_inds            1
  way_miss           89
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           427

CloudFlare speed test :

And immediately after :

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 80107276 bytes 58667 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 800d: dev eth1 root refcnt 2 bandwidth 700Kbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100ms atm overhead 44 memlimit 32Mb
 Sent 4350866 bytes 45683 pkt (dropped 433, overlimits 121884 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 214Kb of 32Mb
 capacity estimate: 700Kbit
 min/max network layer size:           28 /    1500
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:           14

                  Tin 0
  thresh        700Kbit
  target           26ms
  interval        121ms
  pk_delay         76ms
  av_delay       40.4ms
  sp_delay         56us
  backlog            0b
  pkts            46116
  bytes         4567856
  way_inds            0
  way_miss          179
  way_cols            0
  drops             433
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 79919354 bytes 58419 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 800e: dev ifb4eth1 root refcnt 2 bandwidth 14Mbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100ms atm overhead 44 memlimit 32Mb
 Sent 79602499 bytes 57667 pkt (dropped 752, overlimits 90983 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 156Kb of 32Mb
 capacity estimate: 14Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:      106 /    1749
 average network hdr offset:           14

                  Tin 0
  thresh         14Mbit
  target            5ms
  interval        100ms
  pk_delay       9.78ms
  av_delay       6.18ms
  sp_delay         40us
  backlog            0b
  pkts            58419
  bytes        80737220
  way_inds            1
  way_miss          194
  way_cols            0
  drops             752
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           427

Just for my information as I'm curious : which metrics are you looking at ?

moeller0 · January 8, 2024, 7:19pm

Mostly these, these are peak- average- and sparse-delay... (these are updates constantly, so after a speedtest the tests data will "age out" quickly)

this tells you how many times cake dropped a packet (mostly as signal for a flow to slow down) \

Tell you the "threshold" for cake to engage, in your case the minimum sojourn time of packets sent out needs to stay above 26ms for al least 121ms for cake to actually drop something. The way cake/fq_codel work is that during saturating loads delay can increase to be >= target...

What I would do is to set

to 850, as this is roughly what you see as payload rate in the speedtest without SQM, that should recover a bit of throughput and reduce the target a bit as well.

Next thing to do is test your P2P / games updates use case and see whether it is still that bad..

After that I would potentially propose to use https://github.com/moeller0/ATM_overhead_detector to try to actually measure your true per packet overhead (in case this is smaller than the 44 currently applied this might allow to recover a slither of throughput as well, but realistically not all that much).

Simbud · January 8, 2024, 7:29pm

Thanks for explanation, even if I miss some details

I'll try to apply your recommandations and keep the thread updated.

Simbud · January 8, 2024, 10:51pm

Meanwhile, here is the ATM overhead detector result :