SQM on TP-Link Archer C7 R5 - Downloading kills Internet Connection

johannes-z · November 4, 2022, 10:16am

Hello! I wanted to use SQM on my Archer C7 to reduce Bufferbloat and keep Latency stable, but the opposite is happening. It's especially bad when downloading from a faster server such as Steam.

My Internet speeds are advertised as 100Mbit/s DL and 10Mbit/s UL; realistically I reach 90 Mbit/s DL and 10.7 Mbit/s UL.

This is my SQM Config

~# cat /etc/config/sqm

config queue
        option shaper_burst '1'
        option verbosity '5'
        option debug_logging '0'
        option qdisc 'cake'
        option enabled '1'
        option ingress_ecn 'ECN'
        option qdisc_advanced '1'
        option squash_ingress '1'
        option qdisc_really_really_advanced '1'
        option egress_ecn 'NOECN'
        option squash_dscp '1'
        option interface 'eth0.10'
        option eqdisc_opts 'bandwidth 10Mbit besteffort dual-srchost nat rtt 40ms docsis'
        option upload '10000'
        option linklayer 'ethernet'
        option overhead '22'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '0'
        option linklayer_adaptation_mechanism 'default'
        option download '80000'
        option script 'piece_of_cake.qos'
        option iqdisc_opts 'bandwidth 85Mbit besteffort dual-dsthost nat ingress rtt 20ms docsis'

Software Offloading is of course disabled. I use VLAN Tagging for WAN (eth0.10). My PC I'm downloading from is on the same physical Port (eth0.1) - Could this be the problem?

When downloading with SQM enabled my ping start to look like this:

Reply from 216.58.212.131: bytes=32 time=17ms TTL=114
Reply from 216.58.212.131: bytes=32 time=17ms TTL=114
Reply from 216.58.212.131: bytes=32 time=251ms TTL=114
Reply from 216.58.212.131: bytes=32 time=674ms TTL=114
Request timed out.
Reply from 216.58.212.131: bytes=32 time=347ms TTL=114
Reply from 216.58.212.131: bytes=32 time=425ms TTL=114
Reply from 216.58.212.131: bytes=32 time=50ms TTL=114
Reply from 216.58.212.131: bytes=32 time=56ms TTL=114
Reply from 216.58.212.131: bytes=32 time=113ms TTL=114
Reply from 216.58.212.131: bytes=32 time=596ms TTL=114
Reply from 216.58.212.131: bytes=32 time=1541ms TTL=114
Reply from 216.58.212.131: bytes=32 time=45ms TTL=114
Reply from 216.58.212.131: bytes=32 time=437ms TTL=114
Reply from 216.58.212.131: bytes=32 time=22ms TTL=114

TopDog · November 10, 2022, 5:50pm

You might need to lower the upload a bit more, as on VDSL one needs to be 10% to 15% below the unthrottled capacity, so try 9mbps.

Also, on asymmetric lines like that (10:1), you should test using ack-filter on the egress.

moeller0 · November 10, 2022, 6:00pm

These are a bit odd:
a) the rate is already set via option download and option upload so remove one of them (preferably from Xqdisc_opts)
b) setting rtt is not per se a bad idea (if you are sure that all of your network paths are well below the default 100ms), but why set RTT for upload different than for download? This is round trip time so should be the same for both directions.

Yes, that is possible. I assume the C7's eth0 port is connected to a managed switch. Why are doing that? Why not use a single dedicated port for WAN instead?

Could you describe your network toplogy in some more detail, please?

Anf can you post the output of tc -s qdisc, please ?

fkl7834456 · November 10, 2022, 9:26pm

I played with the settings for a long time and the result was always negative, in the end I set it up in my own way, try my settings.
Interface name br-lan
Download speed 200000
Upload speed 200000
and this is based on the fact that your speed is 100 megabits issued by the provider, if it is more, set it more than the Internet speed
Software flow offloading and Hardware thread offload in the Firewall setting, make sure to check the checkboxes )
everything works fine, no loss of internet speed

moeller0 · November 10, 2022, 9:39pm

That tends to be a challenge, for an inward-facing interface the "directions" flip, that is sqm's 'option upload' now actually controlls your download and option download now controls upload....

A flip which does not matter for symmetric configurations like yours. But these configured 200 Mbps will not work well for the OP, either his router runs out of CPU in which case he needs to reduce the shaper rate considerably or he has enough CPU but then setting the shaper to 200 will result in high bufferbloat (unless the access link has low/acceptable bufferbloat by virtue of the ISP configuration).

Erm, software flow offloading should work with SQM (I have not tested it myself) hardware offloading will NOT work with SQM.

I believe that full well, but I am not convinced that SQM does all that much for you.

fkl7834456 · November 11, 2022, 8:47am

with a sharp ping, 5000 does not throw out of the online game, the TV began to work better, there are fewer breaks and freezes and this is what I was striving for and personally tested by me in practice, because theory and practice are different things.
SQM consumes about 4 megabits of RAM, if there is not enough memory, I recommend installing zram

moeller0 · November 11, 2022, 9:19am

I would respectfully argue that in such a case one is better of getting a router with more physical RAM. all memory compression schemes are at best optimizations and all have more or less the same failure modes, depending on the actual input data amount of compression is going to be variable, making the effectiveness in replacing real RAM hard to predict.

I am sorry, I do not understand the point you are making here. What is a "sharp ping" and what is 5000 referring to?

That is true, so testing is essential!

fkl7834456 · November 11, 2022, 9:33am

it happens during an online game, somewhere on the line a heavy load can occur and with a ping of 2000 it will kick you out of the game, there will be a break with the game server, and it doesn’t kick you out of the game, even if the ping grows to 5000, the game still continues to work.
I continued to play for 10 minutes with a ping of 5000, then turned off sqm and I was kicked out of the game and I realized that sqm was working )
in general, sqm does not break the connection, which, in principle, was and is necessary, keeps the connection with the server.
really helps with a bad internet connection and according to my settings the internet speed is maximum, I didn’t notice the load on the router, I’ve been testing for probably a year

johannes-z · November 11, 2022, 10:34am

Alright, I removed the options from Xqdisc_opts. I wasn't sure if I needed both or not - I guess not

I'll have to keep an eye on that, because my ping fluctuates quite a bit, so at times I'm above >20ms. I set the rtt for both to 40ms and try to tweak it in the future.

My Router is in a different physical location than my modem, and I needed to reduce cabling. That's why I use VLAN tagging for WAN/LAN to utilize a single ethernet cable.

Are you sure? I read numerous times, that it doesn't play nicely with SQM.

This is the output after your recommended changes.

~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 466512440705 bytes 559032928 pkt (dropped 2, overlimits 0 requeues 6472)
 backlog 0b 0p requeues 6472
  maxpacket 1514 drop_overlimit 0 new_flow_count 6314 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 809e: dev eth0.10 root refcnt 2 bandwidth 8Mbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 40ms noatm overhead 40 mpu 64
 Sent 15412843 bytes 66250 pkt (dropped 84, overlimits 22040 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 26Kb of 4Mb
 capacity estimate: 8Mbit
 min/max network layer size:           29 /    1500
 min/max overhead-adjusted size:       69 /    1540
 average network hdr offset:           14

                  Tin 0
  thresh          8Mbit
  target         2.28ms
  interval       40.3ms
  pk_delay       6.35ms
  av_delay       3.36ms
  sp_delay         13us
  backlog            0b
  pkts            66334
  bytes        15539999
  way_inds          102
  way_miss          127
  way_cols            0
  drops              84
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           300

qdisc ingress ffff: dev eth0.10 parent ffff:fff1 ----------------
 Sent 146166356 bytes 111498 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 809f: dev ifb4eth0.10 root refcnt 2 bandwidth 80Mbit besteffort dual-dsthost nat wash ingress no-ack-filter split-gso rtt 40ms noatm overhead 40 mpu 64
 Sent 146021302 bytes 110371 pkt (dropped 1127, overlimits 117742 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 148Kb of 4Mb
 capacity estimate: 80Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       86 /    1540
 average network hdr offset:           14

                  Tin 0
  thresh         80Mbit
  target            2ms
  interval         40ms
  pk_delay         29us
  av_delay         17us
  sp_delay          7us
  backlog            0b
  pkts           111498
  bytes       147727328
  way_inds           15
  way_miss          126
  way_cols            0
  drops            1127
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

moeller0 · November 11, 2022, 10:47am

Mind you 100ms is a decent default for the "internet" so this parameter typically does not need changing, the consequence of reducing it is that all flows get earlier congestion feedback, for short RTT flows this can help reducing their intra-flow queuing/delay, for long RTT flows this will result in those flows only being able to achieve less throughput.

That is a bit unfortunate, in that case I think you should instantiate shapers on both eth0.10 and eth0.1 to isolate the two VLANs against each others traffic, but that will result in even more CPU usage. If possible run a second ethernet cable...

No I am not, software flow offloading technically works in a way that is not incompatible with SQM but whether that is true in practice I do not know. Hardware flow offloading however is mostly incomptible with SQM.

Thanks, this looks decent. You still see the issue from above?

johannes-z · November 11, 2022, 11:01am

It's definetly better than befor (generally <100ms), very rare ping timeouts. Bufferbloat is A+ now. Speedtest looks good, but downloading from Steam destroys my poor routers CPU (100% load). I think it's time to upgrade my router

moeller0 · November 11, 2022, 11:06am

I think the C7 class of routers is good for around 70-80 Mbps combined shaping load, you are at 88, so you could try to reduce the download shaper rate to say 40 and retest the CPU usage again. This is not intended as long term solution, but just to confirm your theory that you are already CPU bound.

johannes-z · November 11, 2022, 11:14am

I reduced it to 60Mbps and removed the rtt option, now the ping is around 30ms, and very very rare timeouts. Of course it sucks losing ~40% of my internet speed, but at least it doesn't ruin the fun for everybody in the network
Thanks so much for your help!

moeller0 · November 11, 2022, 11:27am

In the intermediate term... I actually used my wndr3700v2 on a 100/40 link and shaped it down to 49/26 which was the limit it could reliably shape to. For my use-cases the lower rate with SQM resulted in a "better" network, as everything stayed more responsive (I tend to not bother about downloads taking twice as long, as most stuff is either so small I do not notice or so large the download is not instantaneous anyway, and in those cases I am fine with the download time being predictable).

system · November 21, 2022, 11:28am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.