SQM QoS increases download speed but slows down surfing

Dominik-1980 · June 30, 2021, 4:33pm

Hello.

I have installed SQM and put in 85% of my download and upload speed to the luci config.

I have 100MBit/s over Cable.

Without SQM 85 MBit/s. With SQM 98 MBit/s. Great!!

But now the internet pages are loading the pictures very slow.
In the iOS Youtube App for example also if I scroll down it takes about 1 second till the thumbnails appear. The whole surfing is very slow, although the download speed has increased.

Any ideas why and how I could solve this?

It doesn’t matter if I use cake and peace_of_cake or fq_codel…
Downloads are about 15% faster, but surfing the Internet everything is very slow!

moeller0 · June 30, 2021, 4:55pm

This seems rather paradoxical, SQM should never increase the throughput...
Could you please post the output (copy and paste from a terminal window) of the following commands executed after logging in via SSH:

cat /etc/config/sqm
ifstatus wan | grep device
tc -s qdisc

Also it would be great if you could also run a dslreports speedtest both with SQM disabled and enabled as described here.

Dominik-1980 · June 30, 2021, 5:21pm

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option interface 'eth0'
        option upload '3480'
        option debug_logging '0'
        option verbosity '5'
        option download '72000'
        option qdisc 'cake'
        option qdisc_advanced '0'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option enabled '1'

root@OpenWrt:~# ifstatus wan | grep device
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8045: dev eth0 root refcnt 2 bandwidth 3480Kbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 20267123 bytes 29760 pkt (dropped 3511, overlimits 59615 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 177Kb of 4Mb
 capacity estimate: 3480Kbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh       3480Kbit
  target          5.2ms
  interval      100.2ms
  pk_delay        8.5ms
  av_delay        1.8ms
  sp_delay         13us
  backlog            0b
  pkts            33271
  bytes        24660456
  way_inds          460
  way_miss          178
  way_cols            0
  drops            3511
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len          3028
  quantum           300

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 25035021 bytes 36624 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev wlan0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 156733288 bytes 572658 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1023 drop_overlimit 0 new_flow_count 121 ecn_mark 0
  new_flows_len 1 old_flows_len 3
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 128227972 bytes 650168 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1378 drop_overlimit 0 new_flow_count 14770 ecn_mark 0
  new_flows_len 1 old_flows_len 2
qdisc cake 8046: dev ifb4eth0 root refcnt 2 bandwidth 72Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 25547757 bytes 36624 pkt (dropped 0, overlimits 17498 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 26784b of 4Mb
 capacity estimate: 72Mbit
 min/max network layer size:           60 /    1322
 min/max overhead-adjusted size:       60 /    1322
 average network hdr offset:           14

                  Tin 0
  thresh         72Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay        265us
  av_delay         16us
  sp_delay          3us
  backlog            0b
  pkts            36624
  bytes        25547757
  way_inds         1607
  way_miss          226
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            4
  bk_flows            1
  un_flows            0
  max_len          1322
  quantum          1514

Dominik-1980 · June 30, 2021, 5:38pm

Dominik-1980 · June 30, 2021, 5:42pm

Dominik-1980 · June 30, 2021, 5:45pm

dslreports shows weird numbers.

Ookla shows this

Dominik-1980 · June 30, 2021, 5:59pm

I don’t know if it matters… I am using OpenVPN with VyprVPN and their DNS…

dlakelan · June 30, 2021, 6:25pm

It may well matter. What device is SQM actually attached to? The WAN ethernet, or the VPN virtual device?

Dominik-1980 · June 30, 2021, 6:30pm

Okay. I am using 2 routers.
A Fritzbox (192.168.1.1) as main router.
A Raspberry Pi 3B with openWRT as little sister (192.168.1.2) with openVPN as VPN-Router.
The Raspberry ist connected to the Fritzbox with its LAN-Port.

Dominik-1980 · June 30, 2021, 6:38pm

I think I understand.

SQM is actually attached to lan0.

I have lan0, wlan0, br-lan and tun0.
I think I have to choose tun0, right?

Does I have to set 85% of the VPN Speed as download?
Over VPN it is just 30MBit/s instead of 90MBit/s…

moeller0 · June 30, 2021, 6:53pm

Mmmh, that is not what I expected... Could you please post the output of
ifstatus wan (the first tried to reduce the output to lines containing the word "device"), here is what I see:

root@turris:~# ifstatus wan | grep device
	"l3_device": "pppoe-wan",
	"device": "eth2",

That seems not ideal (read wrong). Please try the following replacement for your /etc/config/sqm

	option ingress_ecn 'ECN'
	option egress_ecn 'NOECN'
	option itarget 'auto'
	option etarget 'auto'
	option verbosity '5'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option qdisc_advanced '1'
	option squash_dscp '0'
	option squash_ingress '0'
	option qdisc_really_really_advanced '1'
	option eqdisc_opts 'nat dual-srchost ack-filter'
	option linklayer 'ethernet'
	option linklayer_advanced '1'
	option tcMTU '2047'
	option tcTSIZE '128'
	option linklayer_adaptation_mechanism 'default'
	option debug_logging '1'
	option iqdisc_opts 'nat dual-dsthost ingress'
	option interface 'eth0'
	option overhead '18'
	option tcMPU '64'
	option download '72000'
	option upload '3480'
	option enabled '1'

but I am not 100% sure whether eth0 is the correct interface... and due to VPN the overhead is going to be too small (18 bytes accounts for the overhead the DOCSIS shaper assumes).

moeller0 · June 30, 2021, 7:03pm

For SQM on the Pi3B to actually have a fighting chance, no other machine should be connected to the FritzBox (using the FB for VoIP clients should not matter much, each conversation only takes ~100Kbps, but any computer/mobile device that connects to the internet directly through the FB will make SQM's job harder/impossible).

The dslreports tests report insane retransmission numbers, effortlessly explaining the shitty thoughput...

If all traffic is routed though tun0 than that is where SQM should live.

Yes, and you will also need to figure out the per-packet-overhead from the VPN software and add that to the 18 Bytes I recommend above.

Yes, if your bottleneck only carries 30 Mbps, SQM needs to aim below that number (or get a faster router, I hear a Pi4B should probably run VPN @ 90 Mbps and SQM).

moeller0 · June 30, 2021, 7:07pm

As a starting point, surf over to speedguide.net's TCP analyzer through your VPN and copy and past the contents from the "Share Your Results" box, this might help to figure out the overhead... (or it might not, in which case we need to come up with plan B).

Dominik-1980 · June 30, 2021, 7:13pm

Dominik-1980 · June 30, 2021, 7:17pm

Dominik-1980 · June 30, 2021, 7:25pm

I think I got it now…

First Test is with VPN without SQM.
Second is with VPN with SQM on tun0 and 85% of up- and download-speed of the first test.

Bufferbloat is from B to A+
Quality is from D to A

Up- and Download-Speed is a little bit slower…

I think these are results as expected.

moeller0 · June 30, 2021, 7:39pm

That looks better, but it still is worth getting the overhead roughly correct, see this for the consequences of getting the overhead and/or shaper gross rate wrong.

BTW, for the Uplink I guess it should be possible to go a bit higher without accumulating too much bufferbloat...

Dominik-1980 · June 30, 2021, 8:03pm

Okay.

I think I have now found the best setting.
In conclusion, less bufferbloat makes for lower latency and thus faster surfing at the cost of some bandwidth for larger downloads.

Is that about right?

moeller0 · June 30, 2021, 8:08pm

How does your current /etc/config/sqm look like?

That about sums it up, except you pay the bandwidth cost for all transfers, not just large(r) downloads, but for most use-cases, especially interactive uses that bandwidth sacrifice is more than compensated by the higher responsiveness through the reliably lower latency. Since you seem to understand German, Heise calls this "Schwuppdizitaet" oder gefuehlte Geschwindigkeit (roughly: perceived speed).

Dominik-1980 · June 30, 2021, 8:20pm

I like Schwuppdizitaet!

Yes, I am from Germany.

config queue 'eth1'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option enabled '1'
        option interface 'tun0'
        option download '27000'
        option qdisc_advanced '1'
        option squash_dscp '1'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '0'
        option linklayer 'ethernet'
        option overhead '22'
        option upload '3200'