Setting proper SQM to ensure low latency and jitter

I also missed step #9 for some reason, here it is:

« SpeedGuide.net TCP Analyzer Results » 
Tested on: 2020.03.19 18:00 
IP address: 185.241.xxx.xx 
Client OS/browser: Windows 10 (Firefox 73.0) 
 
TCP options string: 020405a001010402 
MSS: 1440 
MTU: 1480 
TCP Window: 64240 (not multiple of MSS) 
RWIN Scaling: 0 bits  
Unscaled RWIN : 64240 
Recommended RWINs: 63360, 126720, 253440, 506880, 1013760 
BDP limit (200ms): 2570kbps (321KBytes/s)
BDP limit (500ms): 1028kbps (128KBytes/s) 
MTU Discovery: ON 
TTL: 115 
Timestamps: OFF 
SACKs: ON 
IP ToS: 00000000 (0) 

I have also ran a quick test in pingplotter, but I am not sure if I am interpreting this right:

Thanks, BTW you can always edit your existing posts here :wink:

Probably the most convenient options are:
a) install a user friendly editor on the router and use that for editing:
opkg update ; opkg install nano ; nano /etc/config/sqm
Use Ctrl-O to save, and/or Ctrl-X to save and exit.
b) use WinSCP, as winscp will allow you to directly edit files on the router (or so I habe been told)

That confirms/supports the data from tc -s qdisc, your ISP reduces your MTU/MSS by 20 Bytes, which is fine and not a problem in itself, just something to keep in mind.

P.S.: The pingplotter results look decent, best to always/mostly look at latency and loss of the final destination (hop 9 in your case). Intermittent increased latency or loss along the path typically are not diagnostic of anything. If from a specific hop on latency and/or packet loss increase for all later hops this indicates a problematic link along the path (but pinpointing the location exactly is tricky and typically at least requires another traceroute/mtr/pingplotter measurement in the reverse direction)

Really really big thanks for the help, but I was wondering if there is any option to throttle a certain IP adress in my network? Because there are 2 or 3 mobile devices also connected to the WiFi and I don't want them to be considered equal with other hosts if possible. Or maybe throttle the whole WiFi interface a little.
Nonetheless, I applied the SQM config you provided and will test it under pressure tomorrow. My utmost thanks for your time!

Not with SQM, sorry. The nft-qos package offers that, but I never tested that myself.

I hope these are all connecting via the TL-WR1043ND? If not sqm will not work very well as it needs to be able to control/limit all traffic on your link. If yes, you could just instantiate an sqm instance on one of the wifi interfaces (please note that for inward facing network interfaces like WIFI/WLAN/LAN the meaning of the download and upload fields of sqm is inverted*). Or you could switch your WIFI clients to a guest network (see https://openwrt.org/docs/guide-user/network/wifi/guestwifi/guest-wlan-webinterface) that way all wifi clients combined will only get half of the bandwidth if you wire machine is active**. I would believe that is close to what you desire without you needing to track machines individually by IP and/or MAC addresses.

Best Regards

*) Actually the directions are ingress and egress and from an interfaces, that does not change. What does change is how interface ingress\egress are related/aligned to the internet download and upload directions. To make things simpler in the common case of SQM on a WAN interface we named the fields download and upload, so people would intuitively know what to enter; but the flip-side is that people instantiating SQM on inward facing interfaces need to swap the speed settings in relation to the internet/WAN.

**) Actually it would be better to think of this is that your wired machine will get up to half of ingress and egress bandwidth if it actually uses that bandwidth, otherwise the wifi guest will together get all bandwidth that is left over.

Thank you again for the SQM settings - they are doing their job but it is still not enough for me to be satisfied, I have tried to create a new guest network and move all WiFi clients to it, but after configuring everything according to your link, the network itself doesn't work for some reason. Do I need to remove the previous WLAN interface first or? I will try again tho and also I have a question - does the guest network automatically works as you desribed or I need to configure it somehow? ("that way all wifi clients combined will only get half of the bandwidth if you wire machine is active")

Okay, that is odd. I have to admit though, that the last time I set-up a guest network was around 2015 or so, should have mentioned that probably earlier...

It used to be possible to run multiple SSDs on the same radio in paralell, so I assume that the non-guest wifi network does not need to be disabled.

That is a good question, I guess you will need to enable masquerading in the firewall for the guestnetwork somehow... Mmmh, what looked like an easy path is looking like a wild goose chase. Sorry. It still is possible to configure things, but I was looking for something simple and easy to recommend. Not sure how to procede?

I'll probably talk to my ISP tomorrow to maybe increase my bandwith, because there's a possibility of getting additional 5mbps download so maybe that will reduce the problem a little.

Also I found a guide regarding to reducing bufferbloat which practically says to flash your router with dd-wrt
and set up some stuff along with using those 2 commands:

    ifconfig eth0 txqueuelen 2

    ifconfig eth1 txqueuelen 2

I am wondering if this would have any effect on my problem.
Your SQM config works great but I think my problem would be minimised if I would be able to grant at least 50% of the bandwith to myself and leave the other 50% just for WiFi users, because now with fairness applied in SQM, when more than one user will start using WiFi then I am getting less and less with each additional user getting on WiFi.

apply SQM to the wlan0 or whatever your wifi is, set it to an egress maximum of 1/2 the download bandwidth from the ISP... voila, your whole wlan crew get at most put together 1/2 the download... the rest will be for the ethernet

1 Like

Not really, these will instruct a very short hardware queue, but with sqm you basically move the queueing from the hardware queue to a software queue, and manage that software queue well, so that its length is not a problem any more. And since your openwrt router is connected with a fast link to the Ubiquity device, reducing the openwrt router's hardware buffers will not help much at all (you would need to configure the Ubiquity devices buffers to see an effect, but on a bursty MAC like wifi reducing the buffers is trickier than it seems (wifi drivers often have additional layers of buffers) and something requiring balance as otherwise throughput will crater.)

+1; @dlakelan's idea is a great way to implement something now (just make sure to instantiate a shaper each for 2.4 GHz and 5GHz, assuming your router has two wifi bands). It is not the most elegant long-term solution, but it is fast and should let you evaluate quickly whether that type of bandwidth sharing actually helps with your problem.
If it does the easiest way would be to get say an edgerouterX as primary wired router (with OpenWrt) and hook up your TL-WR1043ND as secondary router with NAT enabled, that way all wifi clients will present one and the same IP-address to the cake instance running on the primary router and you get the 1:1 wired:wifi split under load you want, but if the wired machine is not sending receiving the wifi machines will get all available bandwidth (which at your speed seems desirable).

For bonus points, if you are so inclined, you could also use a raspberry pi 4B and a managed switch instead of the edgerouterX, and be prepared for up to gigabit WAN speeds, but at 0.012/0.001.5 Gbps that might be a bit premature...

Yeah I was about to try that, but I am not really sure about the config for that to be the most efficient (qdisc advanced options, link layer adaptation etc.) and if I should remove the per IP fairness on the WAN instance of the SQM?

just do piece of cake everywhere, per IP on wifi and not per ip on wan

Use the same link layer options for all shapers, and also set the ingress keyword for all instances that handle internet ingress traffic (download on WAN, upload for the WLANs and/or LAN).

+1

Alrighty, really big thank you and dlakelan for helping out, I have one more question tho - would changing stuff like checksum offloads etc. on my network adapter would help anything a little? Because it seems like my connection has problems when it comes to sending bigger packets and it probably fragments em very often during gaming, that's why my ping fluctuates horribly probably.

Here's the test I made just now with those 2 SQM instances:

seems pretty good overally for this connection, but If I can somehow combat that bufferbloat a little more, it would be great.

So, let's just do a little math to help you understand the limits of what's possible.

if you have 470kbps upload, then to send a single 1500 byte packet it takes 1500*8 / 470000 = .026 seconds. So, the smallest your jitter will ever get if SQM is tuned perfectly is on the order of 25ms.

Basically, at those speeds, you want to be the only person on the network if you're a gamer. The only way to combat this is to either subscribe to something with about 10x the upload speed, or actually to buy a second line and dedicate it to low-latency communications purposes.

Thanks for clarification dlakelan, well, at the place where I live I have a very limited choice of ISPs, I will try swapping to an ISP who can probably connect me through broadband but we'll have to see if that's even possible. So far thank you for the help again and best of luck to you!

Well, I am resurrecting my topic because I have some more questions and doubts regarding my situation and if some of those could improve anything.

  1. Should I change my MTU size anywhere if ISP itself is reducing it to 1480 and should I change some SQM settings according to it?
  2. Would marking certain packets with DSCP high prio marks improve anything? I need it only for one application.
    I have resolved some issues with my ISP, but ingame I still experience "skipping" from time to time, latency ingame hovers around 50-70ms with no packet loss, but still "skipping" occurs e.g. when sprinting it sometimes teleports me a step/frame backward, feels like a delayed packet.
  3. I have also found a fancy program which you probably know - startrinity continous speed test - as you can see below my uplink is struggling really hard, but first of all I don't even know if that program is accurate and I am reading it right.

You could try to follow https://forum.openwrt.org/t/sqm-cake-and-being-a-twitch-streamer-upload-direction/62474/13 and @StarWhiz's linked blog post to get upstream priority for your gaming packets/gaming host. Please note that for ingress that is much harder....

1 Like

Nah, didn't help at all... My ISP just somehow made their host (to which I am connecting first after my home network) less prone to packet loss, its nearly 0 percent, but my ping jitter is still around +- 100, so it's really unplayable. I have ordered a new router+modem from another ISP to try LTE connection (their BTS is 1.5km from me). I'll try it and if it will look promising, I'll probably get an directional SMA antenna to improve the signal. We'll see in next few days... I am already super tired from trying to get things to at least bearable state so I can just play the game without lagging around, but I cannot see the end of it yet...

003
I was also wondering if this test would help to understand the situation - I have typed in 0.03s packet interval with 280 byte size to somehow imitate the game connection - it seems like the antenna device (3rd host) is the first to delay stuff and increase the latency... I am wondering if showing this to my ISP would help them locate the problem or will I basically make myself look like a fool?

this clearly shows that the device with IP address 192.168.240.1 occasionally causes delays of hundreds of ms as all devices after it have worst case times that are high like that.

if these are wireless devices it may be an inevitable issue with interference or lack of signal strength from your transmitter etc