Strange sqm cake behavior

Installing and Using OpenWrt Network and Wireless Configuration
1

Hi
I've been experiencing strange behaviors with sqm.
I play a lot of online games and especially in dota 2, when i enter a game, i have a ping spike that goes to 600 ms whenever i move my character, if i stay still my ping keeps normal at 60 ms.
When i test my connection in dslreports my bufferbloat is pretty good and i get A or A+.

I tried experimenting with fq_codel + simple.qos, with this my ingame ping doesnt go up when i move my character, but i had a big amount of packet loss.

ifstatus wan:

{
        "up": true,
        "pending": false,
        "available": true,
        "autostart": true,
        "dynamic": false,
        "uptime": 77271,
        "l3_device": "pppoe-wan",
        "proto": "pppoe",
        "device": "eth0",
        "updated": [
                "addresses",
                "routes"
        ],
        "metric": 0,
        "dns_metric": 0,
        "delegation": false,
        "ipv4-address": [
                {
                        "address": "177.177.210.85",
                        "mask": 32,
                        "ptpaddress": "201.79.136.1"
                }
        ],
        "ipv6-address": [

        ],
        "ipv6-prefix": [

        ],
        "ipv6-prefix-assignment": [

        ],
        "route": [
                {
                        "target": "0.0.0.0",
                        "mask": 0,
                        "nexthop": "201.79.136.1",
                        "source": "0.0.0.0/0"
                }
        ],
        "dns-server": [
                "1.1.1.1",
                "1.0.0.1"
        ],
        "dns-search": [

        ],
        "neighbors": [

        ],
        "inactive": {
                "ipv4-address": [

                ],
                "ipv6-address": [

                ],
                "route": [

                ],
                "dns-server": [
                        "200.222.145.84",
                        "200.222.122.180"
                ],
                "dns-search": [

                ],
                "neighbors": [

                ]
        },
        "data": {

        }
}

tc -d qdisc :

qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth1.1 root refcnt 2
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc cake 8037: dev pppoe-wan root refcnt 2 bandwidth 300Kbit diffserv3 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms atm overhead 40
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8038: dev ifb4pppoe-wan root refcnt 2 bandwidth 8200Kbit diffserv3 dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100.0ms atm overhead 40

tc -s qdisc

qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth1.1 root refcnt 2
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc cake 8037: dev pppoe-wan root refcnt 2 bandwidth 300Kbit diffserv3 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms atm overhead 40
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
qdisc cake 8038: dev ifb4pppoe-wan root refcnt 2 bandwidth 8200Kbit diffserv3 dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100.0ms atm overhead 40
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 525479904 bytes 3977424 pkt (dropped 0, overlimits 0 requeues 1)
 backlog 0b 0p requeues 1
  maxpacket 1462 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8037: dev pppoe-wan root refcnt 2 bandwidth 300Kbit diffserv3 dual-srchost nat nowash ack-filter split-gso rtt 100.0ms atm overhead 40
 Sent 392968807 bytes 3546255 pkt (dropped 1465409, overlimits 6730283 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 770Kb of 4Mb
 capacity estimate: 300Kbit
 min/max network layer size:           28 /    1492
 min/max overhead-adjusted size:      106 /    1696
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       18744bit      300Kbit       75Kbit
  target        969.3ms       60.6ms      242.2ms
  interval         1.9s      155.6ms      484.5ms
  pk_delay          0us       78.3ms       40.9ms
  av_delay          0us        3.9ms        8.3ms
  sp_delay          0us         21us        105us
  backlog            0b           0b           0b
  pkts                0      4993710        17954
  bytes               0    502585728      2579790
  way_inds            0       163629            0
  way_miss            0        46906          431
  way_cols            0            0            0
  drops               0        55182            0
  marks               0           15            0
  ack_drop            0      1410227            0
  sp_flows            0            1            2
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         1492          576
  quantum           300          300          300

qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ----------------
 Sent 11344905591 bytes 9394606 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8038: dev ifb4pppoe-wan root refcnt 2 bandwidth 8200Kbit diffserv3 dual-dsthost nat wash ingress no-ack-filter split-gso rtt 100.0ms atm overhead 40
 Sent 10973013952 bytes 9134074 pkt (dropped 260532, overlimits 15196666 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 403Kb of 4Mb
 capacity estimate: 8200Kbit
 min/max network layer size:           28 /    1492
 min/max overhead-adjusted size:      106 /    1696
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh      512496bit     8200Kbit     2050Kbit
  target         35.5ms        5.0ms        8.9ms
  interval      130.4ms      100.0ms      103.9ms
  pk_delay        1.0ms        263us        1.2ms
  av_delay         59us         41us         52us
  sp_delay         36us         21us         36us
  backlog            0b           0b           0b
  pkts              111      9394449           46
  bytes            8397  11344893698         3496
  way_inds            0       561104            0
  way_miss          111        49202           46
  way_cols            0            0            0
  drops               0       260532            0
  marks               0            0            0
  ack_drop            0            0            0
  sp_flows            2            1            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len            76         1492           76
  quantum           300          300          300

with sqm off :http://www.dslreports.com/speedtest/61424304
with sqm on: http://www.dslreports.com/speedtest/61424712

2

I wonder if this is a symptom of having ack filtering enabled?

1 Like
3

It is not, tried both on and off

4

That 300 Kbps uplink is going to be painful, no matter what. If we see how many full MTU packets you can send per second we get 300000 bits/sec / (1696 Bytes/packet * 8 bits/byte) = 22.1108490566 Packets/second, so if you send 22.11*0.6 = 13.266 full MTU packets in a batch, they will "clog" your egress link for 600 milliseconds. Would be interesting to look into packet captures to see what happens when you move your character...

It would also be interesting to see in which tin yor game packets end up in, if they hit the 75Kbps Voice tin, that tin's interval of 484ms and target of 242 might explain some of the pain. Could you test piece_of_cake instead, please? If that is "better" behaved dscp marking might be part of the problem.

5

Looking at these tests, you should be able to set your egress gross shaper speed to at least 400 Kbps (and going from 300 to 400 should help considerably).

6

I am using piece_of_cake right now and the result is the same

If this is the case, what should i do?

My upload varies too much and i fear that raising the value would case problems

7

Mmmh, okay that would rule out an issue by being pushed into the very slow Voice tin...

Well, if you use layer_cake, make sure your network is as quiet as possible then

  1. take a pre-game snapshot of the statistics
    tc -s qdisc > pre-game_tc-statistics.txt

  2. play your game for say 10 minutes or so

  3. take a postgame snapshot of the statistics
    tc -s qdisc > post-game_tc-statistics.txt

post the results here...

Fair enough, it is just that at those rates 100Kbps really matter...