SQM on RPI4 drops considerable amount of speed

CrackedPotato · June 11, 2024, 4:52am

Hello all, went through all the posts regarding SQM and speed and most of them were chalked up to hardware inadequacy, But i am running the latest snapshot on a Raspberry Pi 4, 4GB model, the cpu sits at about 60% on the first core during speedtests with sqm enabled and ethernet with 44 overhead, i got a 300/330MBPS fibre and speedtest reports about 298mbps on the downloads and 330 on upload, I have set the bandwidth to 268 and 291 in the SQM settings but i am getting about 220 and 260 in speedtests sitting next to the router.

I have a UE300 adapter which connects my AP to the RPI4 and the internal nic connects to the ISP modem.

moeller0 · June 11, 2024, 5:42am

Mmmh, am I right that your measurement chain is:
device -> WiFi link -> AP -> ethernet link -> ue300 -> RPi4B -> ethernet link -> ISP router

If yes, maybe try without the WiFi link, and connect the device via ethernet to the AP? WiFi brings in its own set of challenges, so let's characterize sqm performance over ethernet first?

P.S.: Shaper rates are gross rates, so payload/net capacity as measured in online tests is always a few percentage points lower....

CrackedPotato · June 11, 2024, 5:46am

Similar speeds on ethernet, around 230mbps down and 270 up.

moeller0 · June 11, 2024, 6:14am

Maybe you could post the following bits of information to better understand the situation:
check the configuration:

cat /etc/config/sqm
ifstatus wan | grep device
tc -d qdisc
tc -s qdisc
generate some traffic
a full screenshot of the test results of the cloudflare on-line test: https://speed.cloudflare.com
check how sqm responded ot the offered traffic
tc - qdisc # again to see how the test changed the statistics
see what path MTU is in play, for both IPv4 and IPv6, if tracepath is not yet installed run
opkg update , opkg install iputils-tracepath first
tracepath -b 8.8.8.8
tracepath -b 2001:4860:4860::8888

Then it would be good to learn some details about your wan link, is this implemented as ethernet over fiber or via GPON, EPON or XGS-PON?
Also please do this when connected via ethernet...

CrackedPotato · June 11, 2024, 6:40am

Sqm config

config queue 'eth1'
        option enabled '1'
        option interface 'eth0'
        option download '288000'
        option upload '297000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'ethernet'
        option debug_logging '0'
        option verbosity '5'
        option overhead '44'

ifstatus

        "l3_device": "pppoe-wan",
        "device": "eth0",

qdisc noqueue 0: dev lo root refcnt 2
qdisc cake 8005: dev eth0 root refcnt 6 bandwidth 297Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
qdisc cake 8006: dev ifb4eth0 root refcnt 2 bandwidth 288Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8005: dev eth0 root refcnt 6 bandwidth 297Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 1565412574 bytes 2866439 pkt (dropped 565, overlimits 2440041 requeues 493)
 backlog 0b 0p requeues 493
 memory used: 2350592b of 14850000b
 capacity estimate: 297Mbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       60 /    1544
 average network hdr offset:           14

                  Tin 0
  thresh        297Mbit
  target            5ms
  interval        100ms
  pk_delay          3us
  av_delay          1us
  sp_delay          0us
  backlog            0b
  pkts          2867004
  bytes      1566264654
  way_inds          149
  way_miss          421
  way_cols            0
  drops             565
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 4710494627 bytes 3645529 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 7295903274 bytes 5555396 pkt (dropped 0, overlimits 0 requeues 146)
 backlog 0b 0p requeues 146
  maxpacket 1506 drop_overlimit 0 new_flow_count 474 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 2220232123 bytes 3998633 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 7460 drop_overlimit 0 new_flow_count 20 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8006: dev ifb4eth0 root refcnt 2 bandwidth 288Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 4708876833 bytes 3610726 pkt (dropped 34791, overlimits 5548616 requeues 0)
 backlog 18168b 12p requeues 0
 memory used: 11489600b of 14400000b
 capacity estimate: 288Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       90 /    1544
 average network hdr offset:           14

                  Tin 0
  thresh        288Mbit
  target            5ms
  interval        100ms
  pk_delay        734us
  av_delay        609us
  sp_delay         28us
  backlog        18168b
  pkts          3645529
  bytes      4761532033
  way_inds          103
  way_miss          447
  way_cols            0
  drops           34791
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

Do you want the one with the map aswell? since it contaisn my ip and location
https://i.imgur.com/QGAIiHZ.png
https://i.imgur.com/5bNK1EV.png
6.

root@OpenWrt:~# tc - qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8005: dev eth0 root refcnt 6 bandwidth 297Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 1850729874 bytes 4570795 pkt (dropped 568, overlimits 2688569 requeues 1871)
 backlog 0b 0p requeues 1871
 memory used: 2350592b of 14850000b
 capacity estimate: 297Mbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       60 /    1544
 average network hdr offset:           14

                  Tin 0
  thresh        297Mbit
  target            5ms
  interval        100ms
  pk_delay          6us
  av_delay          1us
  sp_delay          1us
  backlog            0b
  pkts          4571363
  bytes      1851586402
  way_inds          159
  way_miss          511
  way_cols            0
  drops             568
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 9009303394 bytes 6562854 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 11582593890 bytes 8452808 pkt (dropped 0, overlimits 0 requeues 165)
 backlog 0b 0p requeues 165
  maxpacket 1506 drop_overlimit 0 new_flow_count 526 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 2468057799 bytes 5702978 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 7460 drop_overlimit 0 new_flow_count 20 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8006: dev ifb4eth0 root refcnt 2 bandwidth 288Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 9019048116 bytes 6508575 pkt (dropped 54279, overlimits 10762163 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 14401216b of 14400000b
 capacity estimate: 288Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       90 /    1544
 average network hdr offset:           14

                  Tin 0
  thresh        288Mbit
  target            5ms
  interval        100ms
  pk_delay        263us
  av_delay        136us
  sp_delay          2us
  backlog            0b
  pkts          6562854
  bytes      9101183350
  way_inds          132
  way_miss          540
  way_cols            0
  drops           54279
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

Starts spaming no reply after some time

1?: [LOCALHOST]                      pmtu 1492
 1:  172.16.64.1 (172.16.64.1)                             5.626ms
 1:  172.16.64.1 (172.16.64.1)                             6.642ms
 2:  dynamic.radinet.in (103.133.120.237)                  9.696ms
 3:  103.170.169.102 (103.170.169.102)                    11.683ms
 4:  72.14.205.240 (72.14.205.240)                        12.515ms asymm  8
 5:  no reply
 6:  no reply
 7:  no reply
 8:  no reply
 9:  no reply
10:  no reply
11:  no reply
^[[D12:  no reply
13:  no reply
14:  no reply
15:  no reply
16:  no reply
17:  no reply
18:  no reply
19:  no reply
20:  no reply
21:  no reply
22:  no reply
23:  no reply
24:  no reply
25:  no reply
26:  no reply
27:  no reply
28:  no reply
29:  no reply
30:  no reply
     Too many hops: pmtu 1492
     Resume: pmtu 1492

dont have ipv6

 1:  send failed
     Resume: pmtu 128000

Its a GPON connection, connected to my isp onu from OPPC model name 2K15X

moeller0 · June 11, 2024, 6:54am

That can be useful to asses wether the test server was not too far away (just tell me roughly how many Km are between your true location and the test server location on the map), but regarding your address all I am interested in is whether this was IPv4 or IPv6 (EDIT, I see below no IPv6 so it must be IPv4 then...). What I do want to see is the latency and packet loss data. So maybe you could take a screenshot of the full page and just redact the map ands IP address portion?

and

Tell me that the best case expected throughput is:

288 * ((1500-8-20-20) / (1500+44)) = 270.8 Mbps
297 * ((1500-8-20-20) / (1500+44)) = 279.3 Mbps

The cloudflare test comes out at 104/123 which means it likely is not actually saturating your link... maybe we need to find a better/more local capacity test to saturate your link...

Thanks!

Mmmmh, thanks for the data, it poses more new questions than it answers...

CrackedPotato · June 11, 2024, 7:05am

https://i.imgur.com/KSBa7cp.png

Test server is about 500km from me

moeller0 · June 11, 2024, 7:22am

Mmmh latency under load looks pretty OK, the packet loss bar is partially obstructed.

35ms delay for 500ms is not great (as a rule of thumb 1ms RTT allows for 100Km of glas fiber, and GPON access latency even under load should be in the range of 2-5ms at worst), but data rarely travels as the crow flies so that might still be fully above board.

CrackedPotato · June 11, 2024, 7:24am

Would you like me to do a ookla speedtest or Netflix fast.com? Ookla has local servers in my city and from my ISP

moeller0 · June 11, 2024, 7:32am

Yes, please both if possible...

CrackedPotato · June 11, 2024, 7:54am

Why does the output look different
Before speedtest

 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 489769719 bytes 884682 pkt (dropped 0, overlimits 0 requeues 29616)
 backlog 0b 0p requeues 29616
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 102597091 bytes 175937 pkt (dropped 0, overlimits 0 requeues 2296)
 backlog 0b 0p requeues 2296
  maxpacket 1514 drop_overlimit 0 new_flow_count 815 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 157202413 bytes 278238 pkt (dropped 0, overlimits 0 requeues 19240)
 backlog 0b 0p requeues 19240
  maxpacket 1514 drop_overlimit 0 new_flow_count 5381 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 108924201 bytes 167371 pkt (dropped 0, overlimits 0 requeues 3427)
 backlog 0b 0p requeues 3427
  maxpacket 1514 drop_overlimit 0 new_flow_count 773 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 90830193 bytes 180631 pkt (dropped 0, overlimits 0 requeues 4644)
 backlog 0b 0p requeues 4644
  maxpacket 1514 drop_overlimit 0 new_flow_count 1411 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 30215821 bytes 82505 pkt (dropped 0, overlimits 0 requeues 9)
 backlog 0b 0p requeues 9
  maxpacket 1462 drop_overlimit 0 new_flow_count 6 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta
rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 1602170814 bytes 1318340 pkt (dropped 0, overlimits 0 requeues 86)
 backlog 0b 0p requeues 86
  maxpacket 1506 drop_overlimit 0 new_flow_count 259 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 15
14 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 1609863 bytes 14585 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0

Speedtest

e Latency:     3.21 ms   (jitter: 0.23ms, low: 2.97ms, high: 3.38ms)
    Download:   193.58 Mbps (data used: 277.2 MB)
                  7.94 ms   (jitter: 9.04ms, low: 3.59ms, high: 229.10ms)
      Upload:   309.74 Mbps (data used: 229.2 MB)
                 14.73 ms   (jitter: 3.65ms, low: 4.62ms, high: 30.10ms)
 Packet Loss:     0.0%

After test

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 738442860 bytes 1101457 pkt (dropped 0, overlimits 0 requeues 30151)
 backlog 0b 0p requeues 30151
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 102956473 bytes 178795 pkt (dropped 0, overlimits 0 requeues 2297)
 backlog 0b 0p requeues 2297
  maxpacket 1514 drop_overlimit 0 new_flow_count 815 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 157560410 bytes 281400 pkt (dropped 0, overlimits 0 requeues 19240)
 backlog 0b 0p requeues 19240
  maxpacket 1514 drop_overlimit 0 new_flow_count 5381 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 169002862 bytes 219508 pkt (dropped 0, overlimits 0 requeues 3459)
 backlog 0b 0p requeues 3459
  maxpacket 1514 drop_overlimit 0 new_flow_count 799 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 277626234 bytes 326365 pkt (dropped 0, overlimits 0 requeues 5146)
 backlog 0b 0p requeues 5146
  maxpacket 1514 drop_overlimit 0 new_flow_count 1732 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target
 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 31296881 bytes 95389 pkt (dropped 0, overlimits 0 requeues 9)
 backlog 0b 0p requeues 9
  maxpacket 1462 drop_overlimit 0 new_flow_count 6 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta
rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 1896557795 bytes 1560810 pkt (dropped 0, overlimits 0 requeues 89)
 backlog 0b 0p requeues 89
  maxpacket 1506 drop_overlimit 0 new_flow_count 271 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 15
14 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 245513922 bytes 231356 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
``|

moeller0 · June 11, 2024, 7:57am

It seems you disabled sqm somehow...

Could you highlight the differences you are concerned about....

CrackedPotato · June 11, 2024, 8:01am

It said disabled in the config for some reason, I renabled it and did a service restart of network, so I am assuming it should be enabled apparent from the drop in speed.

The difference I am referring to is, see the columns of data in the first msg I sent, the columns aren't there right now.

moeller0 · June 11, 2024, 8:15am

Yes, but that is because you did not enable sqm or it errored out when you tried to enable it.... try:
/etc/init.d/sqm stop ; /etc/init.d/sqm start
and post the output here in the thread, please.

CrackedPotato · June 11, 2024, 8:19am

Aaaaaa, got it.

Before test

qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8009: dev eth0 root refcnt 6 bandwidth 297Mbit besteffort triple-isol
ate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 118389 bytes 533 pkt (dropped 0, overlimits 17 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 10880b of 14850000b
 capacity estimate: 297Mbit
 min/max network layer size:           48 /    1431
 min/max overhead-adjusted size:       92 /    1475
 average network hdr offset:           12
                  Tin 0
  thresh        297Mbit
  target            5ms
  interval        100ms
  pk_delay          6us
  av_delay          2us
  sp_delay          1us
  backlog            0b
  pkts              533
  bytes          118389
  way_inds            0
  way_miss           82
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            9
  bk_flows            1
  un_flows            0
  max_len          1445
  quantum          1514
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 385349 bytes 658 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta
rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 1907744955 bytes 1627615 pkt (dropped 0, overlimits 0 requeues 89)
 backlog 0b 0p requeues 89
  maxpacket 1506 drop_overlimit 0 new_flow_count 271 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 15
14 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 257301358 bytes 347186 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 800a: dev ifb4eth0 root refcnt 2 bandwidth 288Mbit besteffort triple-
isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 394561 bytes 658 pkt (dropped 0, overlimits 411 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 68800b of 14400000b
 capacity estimate: 288Mbit
 min/max network layer size:           46 /    1236
 min/max overhead-adjusted size:       90 /    1280
 average network hdr offset:           13
                  Tin 0
  thresh        288Mbit
  target            5ms
  interval        100ms
  pk_delay        749us
  av_delay        140us
  sp_delay         20us
  backlog            0b
  pkts              658
  bytes          394561
  way_inds            0
  way_miss           69
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len          1250
  quantum          1514

Speedtest

Idle Latency:     3.09 ms   (jitter: 0.14ms, low: 2.88ms, high: 3.19ms)
    Download:   179.00 Mbps (data used: 251.9 MB)
                  8.02 ms   (jitter: 4.69ms, low: 3.33ms, high: 231.55ms)
      Upload:   263.75 Mbps (data used: 213.7 MB)
                 10.10 ms   (jitter: 3.19ms, low: 4.57ms, high: 23.52ms)
 Packet Loss:     0.0%

After speedtest

root@OpenWrt:~# tc - qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8009: dev eth0 root refcnt 6 bandwidth 297Mbit besteffort triple-isol
ate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 232408995 bytes 219674 pkt (dropped 16, overlimits 417907 requeues 970)
 backlog 0b 0p requeues 970
 memory used: 1645Kb of 14850000b
 capacity estimate: 297Mbit
 min/max network layer size:           16 /    1500
 min/max overhead-adjusted size:       60 /    1544
 average network hdr offset:           14
                  Tin 0
  thresh        297Mbit
  target            5ms
  interval        100ms
  pk_delay          5us
  av_delay          3us
  sp_delay          2us
  backlog            0b
  pkts           219690
  bytes       232433219
  way_inds          342
  way_miss         1348
  way_cols            0
  drops              16
  marks               0
  ack_drop            0
  sp_flows           10
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
 Sent 274979067 bytes 258429 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta
rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 2183866169 bytes 1885187 pkt (dropped 0, overlimits 0 requeues 89)
 backlog 0b 0p requeues 89
  maxpacket 1506 drop_overlimit 0 new_flow_count 271 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 15
14 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
 Sent 484794710 bytes 566340 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 800a: dev ifb4eth0 root refcnt 2 bandwidth 288Mbit besteffort triple-
isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44
 Sent 278589503 bytes 258424 pkt (dropped 5, overlimits 348236 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 778816b of 14400000b
 capacity estimate: 288Mbit
 min/max network layer size:           46 /    1500
 min/max overhead-adjusted size:       90 /    1544
 average network hdr offset:           14
                  Tin 0
  thresh        288Mbit
  target            5ms
  interval        100ms
  pk_delay          4us
  av_delay          3us
  sp_delay          2us
  backlog            0b
  pkts           258429
  bytes       278597073
  way_inds           53
  way_miss         1254
  way_cols            0
  drops               5
  marks               0
  ack_drop            0
  sp_flows            5
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

moeller0 · June 11, 2024, 9:19am

Upload looks good...

Download considerably worse...

But we see for the download:
before

after

SQM did not really engage here, only 5 drops and no marks for a test running over multiple seconds, so the ´high: 231.55ms´ number, bad as it it seems, is not caused by the router/sqm...
That might imply that you should lower the shaper rate for the download direction,

Which fits with the achieved rate of 179 Mbps, being lower than shaper setting: bandwidth 288Mbit
EDIT: I copied and pasted the upload shaper value initially: bandwidth 297Mbit... that was an accident...

CrackedPotato · June 11, 2024, 9:55am

You mean the upload speed in the sqm settings, right?

Confused because the 297 is upload, which you referred to whole talking about download

CrackedPotato · June 11, 2024, 10:10am

Ok but now for some reason even without SQM the max I am getting with rpi4 is 220, which is far lower than what I get normally.

moeller0 · June 11, 2024, 10:50am

I mixed these up, sorry. The number I am after is for the download:

    Download:   179.00 Mbps (data used: 251.9 MB)
                  8.02 ms   (jitter: 4.69ms, low: 3.33ms, high: 231.55ms)

Both lower throughput that expected and higher latency than desired. The conclusion is the same, SQM did not engage during the download test, which means it did not have control over the relevant bottleneck queue and hence could not meaningfully controll bufferbloat.

That might indicate either an issue with your raspberry pi or your ISPs network to the test servers might be congested, which would effortlessly explain what we see...

BTW, typically we recommend to the the UE 300 as the WAN interface and retain the built in ethernet port as the LAN port, as the U§-300 like other USB ethernet dongles itself has some bufferbloat (which will not matter (much) if you run a traffic shaper on UE-300 as WAN)

CrackedPotato · June 11, 2024, 11:30am

Perhaps something wrong in my configuration? Removing the PI gets me back to 290+ and with the pi running a distro like Ubuntu and connected as a client I get 290+ on the same test.