Making CAKE play nice with Zoom

jnahmias · July 14, 2020, 12:36am

Hello all,

Like most everyone in the time of COVID, I am working from home and thus in many Zoom meetings. I have OpenWrt 19.07.3 installed on my home router / wifi AP which connects to my Cable Modem. I have installed the SQM packages and have enabled CAKE with the layer_cake script giving the following config:

# /etc/config/sqm
config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '3072'
        option upload '1280'
        option qdisc 'cake'
        option script 'layer_cake.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '0'
        option linklayer 'ethernet'
        option verbosity '5'
        option overhead '22'
        option debug_logging '1'
        option squash_dscp '1'
        option squash_ingress '1'

However, despite spending quite a number of hours on Zoom during the day, I see no traffic in the Bulk tin, very little traffic in the Voice tin, and the vast majority in the Best Effort tin:

# tc -s qdisc show dev eth1
qdisc cake 8025: root refcnt 2 bandwidth 1280Kbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 22
 Sent 783760348 bytes 3461979 pkt (dropped 3676, overlimits 2037590 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 328320b of 4Mb
 capacity estimate: 1280Kbit
 min/max network layer size:           28 /    1500
 min/max overhead-adjusted size:       50 /    1522
 average network hdr offset:           14

                   Bulk  Best Effort        Voice
  thresh         80Kbit     1280Kbit      320Kbit
  target        227.1ms       14.2ms       56.8ms
  interval      454.2ms      109.2ms      151.8ms
  pk_delay          0us        5.3ms        1.4ms
  av_delay          0us        501us         84us
  sp_delay          0us         30us         21us
  backlog            0b           0b           0b
  pkts                0      3402508        63147
  bytes               0    777486916     10382523
  way_inds            0       237302         3646
  way_miss            0        78725         8039
  way_cols            0            0            0
  drops               0         3676            0
  marks               0            3            0
  ack_drop            0            0            0
  sp_flows            0            3            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1514          582
  quantum           300          300          300

qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 4748362154 bytes 20533429 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0

I must admit that while I sometimes think I know what I'm doing, this stuff is nearly black magic to me. So, have I missed a step? I thought all I had to do was turn it on and it auto-magically configured itself. Do I need to somehow teach it that Zoom traffic needs to be mapped to Voice?

Thanks in advance for the help,
--Joe

eduardo010174 · July 14, 2020, 9:08am

you need tagging trafic using iptables dscp.

moeller0 · July 14, 2020, 1:34pm

By default zoom does not use dscp marking, this seems a feature for paying customers... You might be able to dscp mark on the machines that actually run the zoom client.

That said, with just 3072/1280 Kbps zoom is not going to be fun... unless you stop all other traffic during calls. But for a cable plan 3.0/1.2 Mbps looks awfully low.

ldir · July 14, 2020, 5:00pm

github.com

ldir-EDB0/sqm-scripts/blob/b8a50d5afaa7c24e6a4b43208e2794f85c0e0210/src/ctinfo_4layercake.qos#L246


#unificontroller
$IPTABLES -t mangle -A QOS_MARK_F_${IFACE} -p tcp -s unificontroller -m tcp -m multiport --dports 8883 -m comment --comment "UnifiController CS0 BE" -j CONNMARK --set-xmark 0x01000000/0x01000000

#Apple
$IPTABLES -t mangle -A QOS_MARK_F_${IFACE} -p tcp -d 17.0.0.0/8 -m tcp -m multiport --dports 5223,993 -m comment --comment "APNS,mail CS0 BE" -j CONNMARK --set-xmark 0x01000000/0x01000000

#Facetime is interesting. Apple say that for firewall purposes only certain ports are used, they don't say if that's source
#or destination. Turns out it's both, so look for connections with both a source & destination port in the Facetime range
ipt -t mangle -A QOS_MARK_F_${IFACE} -p udp -m udp -m multiport --sports 3478:3497,16384:16387,16393:16402 -m multiport --dports 3478:3497,16384:16387,16393:16402,16609:16618 -m comment --comment "Facetime CS3 Vid" -j DSCP --set-dscp-class CS3

#Zoom - connections go to Zoom with dest ports 8801-8810
$IPTABLES -t mangle -A QOS_MARK_F_${IFACE} -p udp -m udp -m set --match-set Zoom4 dst -m multiport --dports 8801:8810 -j DSCP --set-dscp-class CS3 -m comment --comment "Zoom CS3 VI"
$IP6TABLES -t mangle -A QOS_MARK_F_${IFACE} -p udp -m udp -m set --match-set Zoom6 dst -m multiport --dports 8801:8810 -j DSCP --set-dscp-class CS3 -m comment --comment "Zoom CS3 VI"

#Whatsapp Video
ipt -t mangle -A QOS_MARK_F_${IFACE} -p udp -m udp -m multiport --dports 3478 -m comment --comment "Whatsapp CS3 VI" -j DSCP --set-dscp-class CS3

#Irc
ipt -t mangle -A QOS_MARK_F_${IFACE} -p tcp -m tcp -m multiport --dports 6697 -m comment --comment "IRC CS0 BE" -j CONNMARK --set-xmark 0x01000000/0x01000000

#Icmp/v6

So a port & dest IP match, helped by ipsets and dnsmasq resolving & populating that set.

in /etc/config/dhcp
list ipset '/zoom.us/Zoom4,Zoom6'

Zennoe · July 14, 2020, 5:39pm

Off topic: You basically summarised the experience of using computers as a computer scientist.

moeller0 · July 14, 2020, 6:09pm

That might work well, but ports above ~1000 are not reserved guaranteed, so I would recommend to keep an occasional eye on this rule, just to confirm it still is restricted to zoom traffic.

--dports 8801:8810 -j DSCP --set-dscp-class CS3 -m comment --comment "Zoom CS3 VI"

You know this, but CS3 will map to AC_BE on wifi WMM (which in itself is not bad) and will also map into cake's BE tin at least for the OP's layer_cake.qos... So folks, either adjust that to match default cake, or, even better, use @ldir's great ctinfo_4layercake.qos.

jnahmias · July 15, 2020, 12:12am

Thanks all for the feedback!
Ok, as suggested I've copied the ctinfo_4layercake.qos from @ldir and am now using that as my SQM script. Can you expand a bit on this part:

In which section should that go? Is it perhaps accessible from the Luci WUI?

Also, after changing the SQM config to use the ctinfo_4layercake.qos, I got the following in my logs:

Wed Jul 15 00:07:50 2020 user.notice SQM: Starting SQM script: ctinfo_4layercake.qos on eth1, in: 3072 Kbps, out: 1280 Kbps
Wed Jul 15 00:07:50 2020 user.notice SQM: Using generic sqm_start_default function.
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Bulk4 dst -j DSCP --set-dscp-class CS1 -m comment --comment Bulk CS1 ipset
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Bulk4 src -j DSCP --set-dscp-class CS1 -m comment --comment Bulk CS1 ipset
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set BE4 dst -j CONNMARK --set-xmark 0x01000000/0x01000000 -m comment --comment Best Effort CS0 ipset
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set BE4 src -j CONNMARK --set-xmark 0x01000000/0x01000000 -m comment --comment Best Effort CS0 ipset
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Vid4 dst -j DSCP --set-dscp-class CS3 -m comment --comment Vid CS3 ipset
Wed Jul 15 00:07:51 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Vid4 src -j DSCP --set-dscp-class CS3 -m comment --comment Vid CS3 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Voice4 dst -j DSCP --set-dscp-class CS4 -m comment --comment Voice CS4 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Voice4 src -j DSCP --set-dscp-class CS4 -m comment --comment Voice CS4 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Bulk6 dst -j DSCP --set-dscp-class CS1 -m comment --comment Bulk CS1 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Bulk6 src -j DSCP --set-dscp-class CS1 -m comment --comment Bulk CS1 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set BE6 dst -j CONNMARK --set-xmark 0x01000000/0x01000000 -m comment --comment Best Effort CS0 ipset
Wed Jul 15 00:07:52 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set BE6 src -j CONNMARK --set-xmark 0x01000000/0x01000000 -m comment --comment Best Effort CS0 ipset
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Vid6 dst -j DSCP --set-dscp-class CS3 -m comment --comment Vid CS3 ipset
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Vid6 src -j DSCP --set-dscp-class CS3 -m comment --comment Vid CS3 ipset
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Voice6 dst -j DSCP --set-dscp-class CS4 -m comment --comment Voice CS4 ipset
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -m set --match-set Voice6 src -j DSCP --set-dscp-class CS4 -m comment --comment Voice CS4 ipset
Wed Jul 15 00:07:53 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:54 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: FAILURE (2): /usr/sbin/iptables -t mangle -A QOS_MARK_F_eth1 -p udp -m udp -m set --match-set Zoom4 dst -m multiport --dports 8801:8810 -j DSCP --set-dscp-class CS3 -m comment --comment Zoom CS3 VI
Wed Jul 15 00:07:54 2020 user.notice SQM: ERROR: cmd_wrapper: iptables: LAST ERROR: iptables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `iptables -h' or 'iptables --help' for more information.
Wed Jul 15 00:07:54 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: FAILURE (2): /usr/sbin/ip6tables -t mangle -A QOS_MARK_F_eth1 -p udp -m udp -m set --match-set Zoom6 dst -m multiport --dports 8801:8810 -j DSCP --set-dscp-class CS3 -m comment --comment Zoom CS3 VI
Wed Jul 15 00:07:54 2020 user.notice SQM: ERROR: cmd_wrapper: ip6tables: LAST ERROR: ip6tables v1.8.3 (legacy): Couldn't load match `set':No such file or directory  Try `ip6tables -h' or 'ip6tables --help' for more information.
Wed Jul 15 00:07:55 2020 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc filter add dev eth1 protocol all u32 match u32 0 0 action ctinfo dscp 0xfc000000 0x02000000
Wed Jul 15 00:07:55 2020 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory We have an error talking to the kernel, -1
Wed Jul 15 00:07:55 2020 user.notice SQM: WARNING: sqm_start_default: ctinfo_4layercake.qos lacks an egress() function
Wed Jul 15 00:07:56 2020 user.notice SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /usr/sbin/tc filter add dev eth1 parent ffff: protocol all u32 match u32 0 0 action ctinfo dscp 0xfc000000 0x02000000 action mirred egress redirect dev ifb4eth1
Wed Jul 15 00:07:56 2020 user.notice SQM: ERROR: cmd_wrapper: tc: LAST ERROR: RTNETLINK answers: No such file or directory We have an error talking to the kernel, -1
Wed Jul 15 00:07:56 2020 user.notice SQM: WARNING: sqm_start_default: ctinfo_4layercake.qos lacks an ingress() function
Wed Jul 15 00:07:56 2020 user.notice SQM: ctinfo_4layercake.qos was started on eth1 successfully

jnahmias · July 15, 2020, 12:14am

That's a good point on the speeds. I'll have to turn off SQM and retest using dslreports. Maybe later tonight when all is (relatively) quiet.
Ok, I disabled SQM totally and ran five speedtests with the median speeds being:

So, I was off by almost an order of magnitude on the download side; I wonder how that happened... In any case, I then enabled SQM using speeds of 20500/1500 and the standard layer_cake.qos script, ran another five speed tests and got median speeds of:

Though the upload speed came in at anywhere from 0.93 to 1.65 Mbit/s — which is a really wide range to my mind. Any ideas on that?

moeller0 · July 15, 2020, 9:22am

Okay, besides the extreme latency spikes in both down- and upload things look much better, but what mode do you use, could it be one with an intel puma SoC with the infamous latency bug?

Mmh, could you post links ot the individual tests, please?

jnahmias · July 15, 2020, 1:12pm

I'm running OpenWrt on a Netgear WNDR3800. The cable modem reports this:

Model Name: SB6141
Vendor Name: ARRIS Group, Inc.
Firmware Name: SB_KOMODO-1.0.7.0-SCM00-NOSH
Boot Version: PSPU-Boot(25CLK) 1.0.12.18m3
Hardware Version: 7.0
Serial Number: 348781317027075509020017
Firmware Build Time: Jul 22 2016 08:13:57

I don't see it on that bad modems list...
Running the Puma Test from dslreports with SQM enabled (though my connection was not idle) I got:

48ms : xx
49ms : xx
50ms : xx
51ms : xxxx
52ms : xxxx
53ms : xxxxxx
54ms : x
55ms : xxxxx
56ms : xx
57ms : xxx
58ms : xxxxxxx
59ms : xxxxx
60ms : xxx
61ms : xxxxx
62ms : xx
63ms : xxxxx
64ms : xxx
66ms : x
67ms : x
69ms : xxx
72ms : x
73ms : xx
78ms : x
79ms : x
100 - 149ms :xxxx
150 - 199ms :xx
200 - 249ms :x
250 - 299ms :x
300 - 349ms :x
350 - 399ms :x
400 - 449ms :x
450 - 499ms :x
500 - 549ms :xx
550 - 599ms :xx
600 - 649ms :x
650 - 699ms :xx
700 - 749ms :xx
750 - 799ms :x
800 - 849ms :xx
850 - 899ms :x
900 - 949ms :xx
1000 - 1049ms :xx
1050 - 1099ms :x
1100 - 1149ms :x

Looks pretty bad to me...

Here are the tests performed with SQM (layer_cake, speed 20500/1500). All were run with 16 threads, https off, hi-res BB, 60sec duration, avoid compression:

dlakelan · July 15, 2020, 1:19pm

You should try this packet loss and latency test... I'm wondering if maybe your upstream is dropping packets due to physical issues on your line...

https://packetlosstest.com

moeller0 · July 15, 2020, 1:39pm

Thanks, these all look decent, except for the nasty latency spikes... I do not believe that these are caused by your router, but could you try once more with the following in /etc/config/sqm:

config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '20500'
        option upload '1500'
        option qdisc 'fq_codel'
        option script 'simple.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option linklayer 'ethernet'
        option verbosity '5'
        option overhead '22'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '68'
        option linklayer_adaptation_mechanism 'default'
        option debug_logging '1'
        option squash_dscp '1'
        option squash_ingress '1'

This exchanges cake for HTB+fq_codel to test whether the spikes are independent of cake (which I assume they are).

And here my proposal for a tricked out version for your final set-up, that will try to serve all concurrently active internal IP-addresses fairly:

config queue 'eth1'
        option enabled '1'
        option interface 'eth1'
        option download '20500'
        option upload '1500'
        option qdisc 'cake'
        option script 'layer_cake.qos'
        option qdisc_advanced '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost ingress'
        option eqdisc_opts 'nat dual-srchost ack-filter'
        option linklayer 'ethernet'
        option verbosity '5'
        option overhead '22'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option tcMPU '68'
        option linklayer_adaptation_mechanism 'default'
        option debug_logging '1'
        option squash_dscp '1'
        option squash_ingress '1'

jnahmias · July 15, 2020, 1:44pm

Here's what I got:

Total Packet Loss: 0.00%
Upload Packet Loss: 0.00%
Download Packet Loss: 0.00%
Late Packet Rate: 0.00%

Average Latency: 42.86ms
Average Jitter: 4.52ms

Test Settings:
  Duration: 60 seconds
  Frequency: 20 pings/second
  Average Size: 508 bytes
  Acceptable Delay: 100ms

dlakelan · July 15, 2020, 4:19pm

Well, those are good numbers, so you don't have major packet loss issues.

jnahmias · July 16, 2020, 4:07am

Okay, I gave this a shot for three speedtests; seems no better than the layer_cake:

moeller0 · July 16, 2020, 2:08pm

These all suffer from the same set of spurious latency spikes.
At this point, I think you need to start plying with flent's rrul test to figure out whether these spikes are real or just an artifact of how the dslreports speedtest measures latency (the test is decent, but relies on your browser to do the right thing, and test and browser might have differing opinions on what is right...).

But as I expected, no real change from using htb/fq_codel.

One more test you could do, to rule out overload at your docsis plant to set the shaper for 10000/750 basically halving your rates, if this reduces the spiking it might point to queueing at the cmts (but interpretation might be tricky).

jnahmias · July 17, 2020, 4:43am

Okay, I just did a quick flent test both with and without SQM:
flent_rrul-202007170028-SQM_CAKE
flent_rrul-202007170033-noSQM

anon50098793 · July 17, 2020, 6:14am

very interesting... is the last test ( 10000/750 ) consistent?

moeller0 · July 17, 2020, 7:35am

Thanks, so according to flent, things do not look as spiky and dire. If you are willing to share the flent data file I can have a deeper look into them (but the only thing I am a bit puzzled is that the unloaded ICMP latency seems to sits at ~0ms only to jump to ~40ms under load, which is a bit more than I would have expected).
The reduced rate dslreportstest, shows even a massive spie in the idle period at the beginning of the test when sqm would not be doing any shaping, so given the flent results this might be an artifact of running a test in a browser and/or of the test servers that dslreports selected (also, as sidde-note, at 10000/750, 16 streams might be on the high side, which might have caused part of the large Re-xmit
Avg. values for three of the servers).

jnahmias · July 17, 2020, 10:56am

I only ran it once... The network is reasonably quiet, here a couple more runs:

https://pesto.nahmias.net/rrul-2020-07-17T002743.906947.SQM_CAKE.flent.gz
https://pesto.nahmias.net/rrul-2020-07-17T003317.946279.noSQM.flent.gz