Can't get rid of Bufferbloat! (SQM Question)

Bless you gentleman,

I've tried many different settings for SQM to get rid of bufferbloat.
Device: Xiaomi mi a4 gigabit using OpenWRT 21.02.3(Released couple days ago)
Wireguard always on (wireguard client).
MTU is left as wireguard's default 1420 on Wireguard interface.
ISP DL speed 50 mbps - Upload 5 mbps
I guess I have some kind of fiber connection(?), got a very small ONT device connected to my router. It's PPPOE.
Normally I have 100 ms latency to the wireguard server. 130-150 ms to websites.
My latency skyrockets +500 +1-2k under load.

Things I tried for SQM Setup
Download speed (kbit/s) (ingress): 40k, 42,5k
Upload speed (kbit/s) (egress): 4k, 4,25k
piece of cake
Link Layer Adaptation: None
Link Layer Adaptation: Ethernet with overhead
Per Packet Overhead: 44, 84, 104, 124 (also some other values)
Advanced strings (for both ingress and egress): nat, ack-filter, oceanic, satellite, mpu 64, mpu 84

Nothing I did yielded me any better results than keeping SQM off. What gives!?

Test Results: 720P webrtc test for 35 secs (packetlosstest.com)
Without SQM

With SQM

See average latency and jitter increasing with SQM on! The best SQM configuration gave me only close values to SQM off...

This is relatively slow:

a full sized packet around 1500+ (here 1538 for ethernet) bytes will take:
1000*(1538*8)/(5*1000^2) = 2.46 ms when being received
1000*(1538*8)/(0.5*1000^2) = 24.6 ms when being sent.

this is not completely terrible, but that is low enough that you need to manage your expectations, sorry.

So the standard approach would be to try figuring out:
a) what exactly you tried to configure (when using cake the easiest is to run tc -s qdisc and copy and paste the output)
b) what did you do to assess latency under load and from what host (e.g. wired computer, or wireless device connected via WiFi)

these will configure a larger target and interval which wil result in more latency under load.... you really only need those if your experenced RTTs are significantly above the 100ms that cake/fq_codel default to. Increasing this will at best give you a bit more throughput, but certainly increase the "residual bufferbloat".

This only makes real sense on the egress side...

Fixed the main thread about ISP speeds: DL speed 50 mbps - Upload 5 mbps. Don't think it's considered slow? Besides, I can't go lower than 100 ms.

Edit: added test results to main thread.

Edit: testing on wired computer as only LAN device

1 Like

Advanced strings (......)

I've tried WITH or WITOUTH each of them so I wouldn't need to post all of my configuration here. I'm not ssh'ing in the router just using LUCI so I've written down all the setting I've tried.

50/5 while not excessively fast is not slow, I agree.

About the minimum RTT that just means you are network-topologically "far" from the servers you test the RTT against, still the default 'rtt 100' of cake should be decent.

Yeah that is considerably less useful than ssh-ing into the router and running tc -s qdisc as that will also give a few statistics that might help figure out what is going on.

About your test at packetlosstest.com, not sure how well this works. Quite a number of browser based measurement tools reveal more about the browser than necessarily about the network conditions.

So a better test to start out with would be to install and use gping against say 8.8.8.8 while you simultaneously run a speedtest (preferably from a different host).

Here is what I see with the 1080p game stream test:

Or for fun this:

One can only go so far...

image

This is all through WireGuard using NordVPN - and just using LTE connection.

@Avalanche could you also post test result from this website:

Is wireguad active during your tests?
Have you tried without the VPN?

1 Like

It's funny how I'm not seeing those numbers with a wired connection and a PC? Bah!

Are you connecting to a wireguard server which is in your own country?

Yes of course. I only use it to prevent my ISP throttling me. It seems I should really be using a VPS (although I don't understand if that would give the same performance vs NordVPN). I see no appreciable latency/bandwidth difference between not using VPN and using VPN (setting aside the ISP throttling on ports 80 and 443). But then my LTE connection is circa 45ms RTT to first hop and bandwidth between 10Mbit/s and 70Mbit/s.

Is your bandwidth even through VPN and ping to say 8.8.8.8 stable?

Yeah it's stable but SQM makes it worse that's what confused me.