I use a lot of cloud-gaming streaming (Geforce Now, etc) and using my ISPs router/modem wasn't cutting it. I very recently put the modem on bridge-mode, purchased a C7 Archer v5, and flashed OpenWRT. My goal right now is to get a close as possible to zero bufferbloat.
My cable connection is 120/10, and without SQM enabled I get a DSLReports bufferbloat rating of A with maxed-out connection and A+ limited to 50 Mbit, which is obviously a good result. However, if I enable SQM (following the OpenWRT guide, the rating drops to A for 50Mbit. I've played with different settings (cake + piece_of_cake/fq_codel), but SQM always results in a worse bufferbloat-rating.
Again, I realize that an A-rating is a good result, but I'd like to understand why SQM results in worse performance.
So, the dslreports single character bufferbloat rating is a decent help, but for sqm tuning I generally look at the detailed bufferbloat plots in the "Results + Share" page, just click on the links named "Idle", "Downloading", and "Uploading" below the bufferbloat bar graph to get the results for each individual bufferbloat probe. You can also post links to that page here if you want public discussion of your results 
Of course. I'm going to have to split my results into two posts due to the new-user limitation:
SQM On: http://www.dslreports.com/speedtest/55777206
SQM On 50Mbit: http://www.dslreports.com/speedtest/55777258
SQM Off: http://www.dslreports.com/speedtest/55777082
SQM Off 50Mbit: http://www.dslreports.com/speedtest/55777332
Looking at these results, I can see now that SQM definitely does have a positive effect when it comes to maxed downstream and upload. In these tests I did not get an A+ for SQM-disabled at 50Mbit, but I was previously. The reason why I'm preoccupied with the 50Mbit results is that's the max bitrate for Geforce Now.
These results look a bit odd, could you please post the output of:
cat /etc/config/sqm
tc -s qdisc' tc -d qdisc`
Also, how did you limit to 50 Mbit without SQM?
I was using the advanced options on the DSLReports speed test to limit the speed to 50 Mbit. I'm not sure if that causes complications. Here's the output of those commands with SQM enabled. Thanks for the help!
root@OpenWrt:~# cat /etc/config/sqm
config queue 'eth1'
option verbosity '5'
option linklayer 'ethernet'
option download '100000'
option interface 'br-wan'
option qdisc 'cake'
option script 'piece_of_cake.qos'
option debug_logging '1'
option upload '9000'
option qdisc_advanced '0'
option overhead '24'
option enabled '1'
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 24615435326 bytes 33407427 pkt (dropped 0, overlimits 0 requeues 63)
backlog 0b 0p requeues 63
maxpacket 1514 drop_overlimit 0 new_flow_count 73 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 803b: dev br-wan root refcnt 2 bandwidth 9Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 24
Sent 4147060 bytes 33722 pkt (dropped 5, overlimits 4984 requeues 0)
backlog 0b 0p requeues 0
memory used: 36960b of 4Mb
capacity estimate: 9Mbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 52 / 1524
average network hdr offset: 14
Tin 0
thresh 9Mbit
target 5.0ms
interval 100.0ms
pk_delay 1.3ms
av_delay 83us
sp_delay 6us
backlog 0b
pkts 33727
bytes 4154091
way_inds 144
way_miss 3175
way_cols 0
drops 5
marks 0
ack_drop 0
sp_flows 51
bk_flows 1
un_flows 0
max_len 11632
quantum 300
qdisc ingress ffff: dev br-wan parent ffff:fff1 ----------------
Sent 114069975 bytes 887240 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.2 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 803c: dev ifb4br-wan root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100.0ms noatm overhead 24
Sent 126483227 bytes 887234 pkt (dropped 6, overlimits 63117 requeues 0)
backlog 0b 0p requeues 0
memory used: 321408b of 5000000b
capacity estimate: 100Mbit
min/max network layer size: 46 / 1500
min/max overhead-adjusted size: 70 / 1524
average network hdr offset: 14
Tin 0
thresh 100Mbit
target 5.0ms
interval 100.0ms
pk_delay 11us
av_delay 4us
sp_delay 2us
backlog 0b
pkts 887240
bytes 126491335
way_inds 141
way_miss 3051
way_cols 0
drops 6
marks 0
ack_drop 0
sp_flows 1
bk_flows 1
un_flows 0
max_len 1514
quantum 1514
root@OpenWrt:~# tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
qdisc cake 803b: dev br-wan root refcnt 2 bandwidth 9Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 24
qdisc ingress ffff: dev br-wan parent ffff:fff1 ----------------
qdisc noqueue 0: dev eth0.2 root refcnt 2
qdisc noqueue 0: dev wlan0 root refcnt 2
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc cake 803c: dev ifb4br-wan root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat > > wash no-ack-filter split-gso rtt 100.0ms noatm overhead 24
Interesting, did not realize the test offered that option, but who knows how they implemented that feature internally, so I guess the 5 Mbit plots will tell as much about your router as about dslreports.
What is the output of ifstatus wan ?
What is the content of the "Share your Results" box on https://www.speedguide.net/analyzer.php?
root@OpenWrt:~# ifstatus wan
{
"up": true,
"pending": false,
"available": true,
"autostart": true,
"dynamic": false,
"uptime": 151959,
"l3_device": "br-wan",
"proto": "dhcp",
"device": "br-wan",
"updated": [
"data"
],
"metric": 0,
"dns_metric": 0,
"delegation": true,
"ipv4-address": [
{
"address": "MyIP",
"mask": 22
}
],
"ipv6-address": [
],
"ipv6-prefix": [
],
"ipv6-prefix-assignment": [
],
"route": [
{
"target": "0.0.0.0",
"mask": 0,
"nexthop": "NextHopIP",
"source": "MyIP\/32"
}
],
"dns-server": [
"64.59.168.13",
"64.59.174.84"
],
"dns-search": [
"ok.shawcable.net"
],
"inactive": {
"ipv4-address": [
],
"ipv6-address": [
],
"route": [
],
"dns-server": [
],
"dns-search": [
]
},
"data": {
"hostname": "S0106b0be76776553",
"leasetime": 172800
}
}
« SpeedGuide.net TCP Analyzer Results »
Tested on: 2019.10.27 17:40
IP address: 174.4.xxx.xx
Client OS/browser: Windows 10 (Chrome 77.0.3865.120)
TCP options string: 020405b40103030801010402
MSS: 1460
MTU: 1500
TCP Window: 262656 (not multiple of MSS)
RWIN Scaling: 8 bits (2^8=256)
Unscaled RWIN : 1026
Recommended RWINs: 64240, 128480, 256960, 513920, 1027840
BDP limit (200ms): 10506kbps (1313KBytes/s)
BDP limit (500ms): 4202kbps (525KBytes/s)
MTU Discovery: ON
TTL: 111
Timestamps: OFF
SACKs: ON
IP ToS: 00000000 (0)
Mmmh, thanks, br-wan sounds like a bridge interface ad these often are tricky for sqm, but according to ifstatus that is correct.
And the speedguide.net result seems to rule out ds-lite, need to think a bit more about your config...
My modem is in bridge-mode, if that's what you mean. Is there another configuration that would be more ideal?
Sorry, for being unclear, in openwrt parlance the be in br-wan indicates a bridges interface. Now for your router that seems to be correct, but it still might cause issues, for example bridges interfaces often do not generate hotplug events like simple interfaces which can cause issues with sqm.
A modem in bridged mode however should not be a problem.
Just as a follow-up, conparing both the SQM on and sqm off dslreports bufferbloat plots in detail, it becomes clear that the sqm results are more consistent in that they show much less variation of the lag/RTT bufferbloat probes. The theoretical limits from your configuration are:
100000 * (1500-20-20) / (1524) = 95.800 Mbps
9000 * (1500-20-20) / (1524) = 8.622 Mbps
So the 90.6/8.76 that you get seem to be okay-ish, especially since IIRC your router's CPU will be hard pressed to shape above ~70-100 Mbps anyway.
Nope, bridge mode is fine, I guess you would need a more powerful router to get closer to your link's real limit of 122/11 Mbps*, but bufferbloat-wise the plots look fine and I would not bother about the difference in the bufferbloat grade from A+ to A, it is notoriously hard to compress the relevant data into a single qualifier, and IMHO looking at the time resolved bufferbloat plots indicates that sqm does a decent job on your link keeping latency under load in check.
*) That is my hypothesis, but I might be wrong and you could try to set
option download '120000'
option upload '11000'
And run the speedtest again...
In addition I would also add:
option qdisc_advanced '1'
option qdisc_really_really_advanced '1'
option iqdisc_opts 'nat dual-dsthost ingress'
option eqdisc_opts 'nat dual-srchost ack-filter'
This will:
a) shape ingress traffic such that the shaper takes its position on the wrong end of the bottleneck into account and will try to shape in a way that traffic on the bottleneck link is shaped to the set rate instead of the egress of the shaper (ingress)
b) will look at the internal IP addresses of all packets (that is the destination address for download/ingress traffic dual-dsthost, and the source address for upload/egress traffic dual-srchost); the nat ketyword will instruct cake to look into the conntrack tables to get reliable access to the internal endpoint IP addresses
c) will intelligently prune pure ACK packets in the egress queue by essentially sending only the most recent ACK packet (if multiple eligible ACK packets for the same flow are queued up in the first place), this will a) reduce the share of egress bandwidth that is used for ACK traffic and due to the burstyness of the request-grant cycle for egress traffic on a docsis link will not even increase the time it takes for the ACK signal to reach the remote endpoint. (A number of docsis ISP does this already silently in their modems).
The first two options really are only requured so that the other values are not cleared if you open luci-app-sqm.
Good luck.
1 Like