GL-MT3000 SQM performance test

Hardware Questions and Recommendations
1

I tested GL-MT3000 SQM performance using these default settings, except for the link speed:

config queue 'eth0'
        option enabled '1'
        option interface 'eth0'
        option download '300000'
        option upload '300000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option debug_logging '0'
        option verbosity '5'

I have 350/350 ETTH. I used Waveform Bufferbloat test and htop for CPU usage.

SQM latency speed CPU usage Notes
disabled 9/11/0 352.2/356.1 15-30% sometimes latency is much higher
enabled with default linklayer 9/0/0 288.6/285.6 55-70%
overhead 38 mpu 84 noatm nat 7/2/1 282.2/284.6 55-70%
overhead 38 mpu 84 noatm 8/0/0 286.9/285.5 55-70%
download/upload 340000 9/5/0 334.2/330.1 55-70%
download/upload 340000 + overhead 38 mpu 84 noatm nat 9/4/0 322.2/329.1 55-70%

overhead 38 mpu 84 noatm taken from https://man7.org/linux/man-pages/man8/tc-cake.8.html#OVERHEAD_COMPENSATION_PARAMETERS for ethernet. Default is raw overhead 0. What effect should I expect with/without the overhead set?

GL-MT3000 #1
Strange results here. It seems test #2 was more in line with other MT7981 devices.
I'm not sure how relevant is testing SQM on lan. With the following setup:

  • SQM up/down set to 900Mbps
  • GL-MT6000 running iperf3 -s (1)
  • GL-MT3000 wan 2.5G port connected to 2.5G or 1G lan port of GL-MT6000
  • GL-MT3000 is doing NAT
direction speed retries CPU usage
client -> server 844 8 one core at ~100%
server -> client (-R) 491 1 both CPU cores 60-70%

GL-MT3000 #2

  • SQM up/down set to 900Mbps
  • x86_64 NAS running iperf3 -s and connected to 1G lan port of GL-MT6000
  • GL-MT3000 wan 2.5G port connected to 1G lan port of GL-MT6000
  • GL-MT3000 is doing NAT
direction speed retries CPU usage
client -> server 519 1 one core at ~100%
server -> client (-R) 650 0 one core at ~100%

With linklayer: Ethernet overhead: 38 mpu: 84

direction speed retries CPU usage
client -> server 610 1 one core at ~100%
server -> client (-R) 668 6 one core at ~100%
2

See
https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details#sqmlink_layer_adaptation_tab
Which contains a discussion on how the per-packet-overhead setting and the gross shaper rate are not fully independent control variables, especially if the overhead is set too low, the traffic shaper will allow more than gross-shaper-rate to escape if the traffic consists out of small packets...

3

Thank you for the explanation. I looked into iperf3 man and the only thing that seems to be close is --set-mss 100. Is there any better test for small packet bufferbloat?

4

You could also try MSS clamping in your router, but not all operating systems will accept arbitrarily MSS values, I think Macos allowed up to 216 or so in the past.
Also keep in mind that some part of the cost in a router is per packet so if you reduce the MTU from 1500 to 150 (assuming your OS will not ignore that setting) during a capacity test your router and its network stack will have to handle ~10 times as many packets. If the router is not 'beefy' enough to do that you will less throughput than expected and with cake also higher delays...

5

The only way I could find any difference between various overhead and mtu values was with flent graphs. Best was overhead 38 and mtu 84 (the same values are in cake man page for plain ethernet). I observed the least sawtooth patterns with these settings. However, my ISP has bad routing or server is far enough that testing with full speed wasn't a good idea. Only reducing up/down limits allowed for overhead tuning and other experiments. Then I raised these limits again based on speed.cloudflare.com which has a POP nearby.

Another thing I noticed is that testing with ethernet should be just one part of SQM performance evaluation. Turns out GL-MT3000 would need much more CPU power to handle these levels of SQM processing when lan side is connected by WiFi. WiFi (2 streams, AX, laptop next to the router) takes so much CPU time, that cake chokes and I see lower performance than with ethernet. irqbalance helped a bit as all interrupts were handled by core 0.

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   387 MBytes   325 Mbits/sec    6             sender
[  5]   0.00-10.00  sec   386 MBytes   324 Mbits/sec                  receiver

With -R:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   389 MBytes   326 Mbits/sec    5             sender
[  5]   0.00-10.00  sec   385 MBytes   323 Mbits/sec                  receiver

It seems that enabling WED is reducing these numbers somewhat, or at least not helping:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   351 MBytes   294 Mbits/sec    6             sender
[  5]   0.00-10.01  sec   350 MBytes   293 Mbits/sec                  receiver

With -R:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   385 MBytes   323 Mbits/sec    3             sender
[  5]   0.00-10.00  sec   381 MBytes   320 Mbits/sec                  receiver
6

Out of curiosity: could you also run a test using fq_codel / simple.qos ?

7

Ok, I’ll test with fq_codel one day.

1 Like
10

Hello. I am the owner of a mt3000 router and the question is how can I set SQM, I mention that I am new to the openwrt domain and this is my first router. I have firmware from gl.inet, with the latest update. The internet bandwidth is 1000 mbps.

11

Start here:
https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm
then follow up here:
https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details

If you have specific question, just ask here in the forum.