New to SQM / looking for help

Installing and Using OpenWrt
1

New guy here. Moved to openwrt from the netduma r2, as it never worked correctly. My bufferbloat test show I have over 100 ms of latency.

I decided to install openwrt in an older i5 pc, all looks great but for some reason I cannot get SQM to work correctly.

Here is what I have:

Dell 3040 pc/ I5 /8gb/128gb ssd /non managed powered 8 port gigabit switch

Openwrt version generic-ext4-combined-efi.img.gz

WOW 1gb cable internet (only get about 850 down and 50 up)

I followed all of the FAQs to set up SQM (https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm), but I am still getting the same latency and in some cases higher. I adjusted my speed as low as 400000 down and 30000 up and still it sees it gets worse.

I changed from the default discipline to cake and it got really worse…which confused me because from my days of research it shows is supposed to be better.

Frist thanks to the community for openWRT, amazing stuff.

If any one has any suggestions please let me know.

2

Mmmh, maybe start off with describing your setup:
Does the PC use two network interfaces or are you using VLANs to multiplex WAN ans LAN access via a single ethernet interface?

Could you please post the output of the following commands (run them on the router's command line interface and copy and paste the contents here with proper formatting*):

cat /etc/config/sqm
tc -s qdisc
ifstatus wan

that might (or might not) give us some clues what might be going on or at least whereto look next.

*)

Please use the "Preformatted text </>" button for logs, scripts, configs and general console output.
grafik
Please edit your post accordingly. Thank you! :slight_smile:

3

Sorry about the delay...

I set the built in adaptor as the br / lan
I have a USB 3.0 1gb ethernet as the wan

here is the results:

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option interface 'eth1'
        option debug_logging '0'
        option verbosity '5'
        option enabled '1'
        option linklayer 'none'
        option download '700000'
        option upload '45000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'


root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 207270786346 bytes 144833694 pkt (dropped 91, overlimits 0 requeues 338564)
 backlog 0b 0p requeues 338564
  maxpacket 1514 drop_overlimit 0 new_flow_count 734051 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8057: dev eth1 root refcnt 2 bandwidth 45Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 856440809 bytes 3869673 pkt (dropped 144, overlimits 1481690 requeues 1)
 backlog 0b 0p requeues 1
 memory used: 675712b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         45Mbit
  target            5ms
  interval        100ms
  pk_delay       1.12ms
  av_delay        133us
  sp_delay          1us
  backlog            0b
  pkts          3869817
  bytes       856628418
  way_inds        81416
  way_miss       106843
  way_cols            0
  drops             144
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len         23136
  quantum          1373

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 30388852418 bytes 27355190 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8058: dev ifb4eth1 root refcnt 2 bandwidth 700Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 30771777912 bytes 27355157 pkt (dropped 33, overlimits 32673135 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1812314b of 15140Kb
 capacity estimate: 700Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        700Mbit
  target            5ms
  interval        100ms
  pk_delay         66us
  av_delay         16us
  sp_delay          1us
  backlog            0b
  pkts         27355190
  bytes     30771825078
  way_inds      3178234
  way_miss       106215
  way_cols            0
  drops              33
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 207271496032 bytes 144835019 pkt (dropped 91, overlimits 0 requeues 338569)
 backlog 0b 0p requeues 338569
  maxpacket 1514 drop_overlimit 0 new_flow_count 734057 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8057: dev eth1 root refcnt 2 bandwidth 45Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 857157664 bytes 3870829 pkt (dropped 144, overlimits 1482199 requeues 1)
 backlog 0b 0p requeues 1
 memory used: 675712b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         45Mbit
  target            5ms
  interval        100ms
  pk_delay        310us
  av_delay         78us
  sp_delay          2us
  backlog            0b
  pkts          3870973
  bytes       857345273
  way_inds        81416
  way_miss       106862
  way_cols            0
  drops             144
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len         23136
  quantum          1373

qdisc ingress ffff: dev eth1 parent ffff:fff1 ----------------
 Sent 30389601423 bytes 27357807 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8058: dev ifb4eth1 root refcnt 2 bandwidth 700Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 30772563555 bytes 27357774 pkt (dropped 33, overlimits 32673690 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1812314b of 15140Kb
 capacity estimate: 700Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        700Mbit
  target            5ms
  interval        100ms
  pk_delay          6us
  av_delay          4us
  sp_delay          1us
  backlog            0b
  pkts         27357807
  bytes     30772610721
  way_inds      3178234
  way_miss       106234
  way_cols            0
  drops              33
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

root@OpenWrt:~#


root@OpenWrt:~# ifstatus wan
Interface wan not found
root@OpenWrt:~#
4

here is a test I just did. something is not working for sure...

bufferbloat
bufferbloat1540×644 103 KB

5

This is not correct (you probably need to account for 18 bytes) but also not your issue.

Ah these are not all equally usable for high rates, what make and manufacturer is this? Could you switch pt a real NIC instead?

6

OK i changed link layer adaptation to: ethernet with overhead and PPO to 18, just incase :slight_smile:
My usb adaptor is
Startech USB 3.0 to Gigabit Ethernet NIC Network Adapter
chipset ASIX - AX88179
The pc is a micro ATX, so no expansion slots...should I switch the BR/lan to the USB and use the built to WAN?

7

Ah, we are getting somewhere, the ASIX ethernet dongles are rumoured to not allow close to gigabit rates at least not with Linux. The TP-Link UE300, based on a realtek chip has the reputation of allowing gigabit rates however, so maybe time to change that dongle?

1 Like
8

Ok interesting. I do have another PC with a built in gigabit nic and also an 4 port Intel I340-T4 card. Ill try that tonight and see if I see any changes.

Thanks

1 Like
9

Posting here again to see if I can some help in figuring this out.

So I decided to follow the suggestions in here and move away from the USB ethernet adaptor.

I have a pc with a built in Nic (gigabit) and I also added an intel 340 4 port card ( all gigabit)

WAN is the local nic / and then the BR/LAN is composed of the other 4. Also have a powered 8 port, also all gigabit.

The pc using to test is connected straight to the switch then switch to the pc (openwrt wan).

If I set my bandwidth at 80% (650000) up and down, I get a bunch of latency over 100, almost the same as if SQM was disabled. if asset it at 50%, I get about the same…only when I got below 200000 I can see the latency lowering.

Also when switching from cake to fqcodel it seems to get better, but still too high.

Not sure if its just my provider/something configured wrong…

here is my output

OpenWrt 21.02.1, r16325-88151b8303
 -----------------------------------------------------
=== WARNING! =====================================
There is no root password defined on this device!
Use the "passwd" command to set up a new password
in order to prevent unauthorized SSH logins.
--------------------------------------------------
root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option interface 'eth4'
        option debug_logging '0'
        option verbosity '5'
        option enabled '1'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'none'
        option download '750000'
        option upload '45000'

root@OpenWrt:~#

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 21275443381 bytes 15717835 pkt (dropped 0, overlimits 0 requeues 279)
 backlog 0b 0p requeues 279
qdisc fq_codel 0: dev eth0 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 2407607603 bytes 1697185 pkt (dropped 0, overlimits 0 requeues 73)
 backlog 0b 0p requeues 73
  maxpacket 66616 drop_overlimit 0 new_flow_count 21 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 2388645005 bytes 1726301 pkt (dropped 0, overlimits 0 requeues 54)
 backlog 0b 0p requeues 54
  maxpacket 51476 drop_overlimit 0 new_flow_count 24 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 3387578915 bytes 2448715 pkt (dropped 0, overlimits 0 requeues 20)
 backlog 0b 0p requeues 20
  maxpacket 3028 drop_overlimit 0 new_flow_count 26 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 3343585518 bytes 2555917 pkt (dropped 0, overlimits 0 requeues 68)
 backlog 0b 0p requeues 68
  maxpacket 56018 drop_overlimit 0 new_flow_count 29 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 2102223632 bytes 1569709 pkt (dropped 0, overlimits 0 requeues 6)
 backlog 0b 0p requeues 6
  maxpacket 1514 drop_overlimit 0 new_flow_count 4 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 2250138746 bytes 1612190 pkt (dropped 0, overlimits 0 requeues 22)
 backlog 0b 0p requeues 22
  maxpacket 1514 drop_overlimit 0 new_flow_count 13 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 3907539715 bytes 3018899 pkt (dropped 0, overlimits 0 requeues 33)
 backlog 0b 0p requeues 33
  maxpacket 1514 drop_overlimit 0 new_flow_count 26 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 1488124247 bytes 1088919 pkt (dropped 0, overlimits 0 requeues 3)
 backlog 0b 0p requeues 3
  maxpacket 1514 drop_overlimit 0 new_flow_count 3 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth2 root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth2 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth3 root
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth3 parent :8 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :7 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :6 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :5 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8015: dev eth4 root refcnt 2 bandwidth 45Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 855768505 bytes 3497932 pkt (dropped 703, overlimits 3257715 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1504304b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         45Mbit
  target            5ms
  interval        100ms
  pk_delay       1.06ms
  av_delay        300us
  sp_delay          8us
  backlog            0b
  pkts          3498635
  bytes       856786645
  way_inds       232433
  way_miss        68902
  way_cols            0
  drops             703
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len         30280
  quantum          1373

qdisc ingress ffff: dev eth4 parent ffff:fff1 ----------------
 Sent 14714358907 bytes 14900988 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8016: dev ifb4eth4 root refcnt 2 bandwidth 750Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 15076974597 bytes 14899105 pkt (dropped 1882, overlimits 11968633 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 15170998b of 15140Kb
 capacity estimate: 750Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        750Mbit
  target            5ms
  interval        100ms
  pk_delay         59us
  av_delay         13us
  sp_delay          2us
  backlog            0b
  pkts         14900987
  bytes     15079805511
  way_inds       117328
  way_miss        68344
  way_cols            0
  drops            1882
  marks               0
  ack_drop            0
  sp_flows            2
  bk_flows            1
  un_flows            0
  max_len         66616
  quantum          1514

root@OpenWrt:~#
root@OpenWrt:~# ifstatus wan
Interface wan not found
root@OpenWrt:~#

root@OpenWrt:~# ifconfig
br-lan    Link encap:Ethernet  HWaddr 00:1B:21:BE:DA:68
          inet addr:192.168.10.1  Bcast:192.168.10.255  Mask:255.255.255.0
          inet6 addr: fe80::21b:21ff:febe:da68/64 Scope:Link
          inet6 addr: fddc:b661:ce56::1/60 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5151163 errors:0 dropped:0 overruns:0 frame:0
          TX packets:13923417 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1279919680 (1.1 GiB)  TX bytes:21196294244 (19.7 GiB)

eth0      Link encap:Ethernet  HWaddr 00:1B:21:BE:DA:68
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:5388493 errors:0 dropped:60 overruns:0 frame:0
          TX packets:15740011 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:1365529200 (1.2 GiB)  TX bytes:21295194630 (19.8 GiB)
          Memory:f7200000-f727ffff

eth1      Link encap:Ethernet  HWaddr 00:1B:21:BE:DA:69
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:f7100000-f717ffff

eth2      Link encap:Ethernet  HWaddr 00:1B:21:BE:DA:6A
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:f7080000-f70fffff

eth3      Link encap:Ethernet  HWaddr 00:1B:21:BE:DA:6B
          UP BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)
          Memory:f7000000-f707ffff

eth4      Link encap:Ethernet  HWaddr 48:4D:7E:FD:3B:DE
          inet addr:74.199.126.96  Bcast:74.199.127.255  Mask:255.255.254.0
          inet6 addr: fe80::4a4d:7eff:fefd:3bde/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:19441480 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5374840 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:21667632620 (20.1 GiB)  TX bytes:1381556332 (1.2 GiB)
          Interrupt:19 Memory:f7480000-f74a0000

ifb4eth4  Link encap:Ethernet  HWaddr E6:B8:8E:BF:E3:02
          inet6 addr: fe80::e4b8:8eff:febf:e302/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:14927311 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14927311 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32
          RX bytes:15096503968 (14.0 GiB)  TX bytes:15096503968 (14.0 GiB)

lo        Link encap:Local Loopback
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:3461 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3461 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:297905 (290.9 KiB)  TX bytes:297905 (290.9 KiB)

Thanks :slight_smile:

10

First things first ;), the router is telling you something here, please follow its recommendation.

What does that mean? Your cable link is asymmetric so 650000 (650 Mbps) might work fir the download/ingress direction, but will be too large for the upload/egress, so which download/upload combination did you actually test? It is easiest to just post a link to the respective test result and post the output of tc -s qdisc show dev eth4 and tc -s qdisc show dev ifb4eth4 here as well.

This is a bit unexpected, typically this helps figuring out which interface is acting as WAN, but I assume that eth4 is the correct one? (It at least gets an IPv4 address (74.199.126.96) out of WOW's range, and the traffic counters indicate your traffic comes in via eth4 and takes eth0 into your LAN).

But web based speedtests can be inaccurate at times, so maybe it is time to run a speedtest (e.g. fast.com) while simultaneously running the following on the router:
mtr -ezb4 8.8.8.8
(you might need to install mtr first: opkg update ; opkg install mtr) and look how the latency to the final target behaves, before, during and after the test.

11

First thanks for replying, I do appreciate your help ...

password set :slight_smile: ...silly of me

sorry for the misinformation, my download is set to 750 right now and 45 down. i played with this all night. and the only way i was able to get almost 0 latency was by using 200000 down and 30000 up. although it seems my upload is not getting affected by bufferbloat somehow...but 200000 limit seems too low to outweight the benefits...which is why i am wondering id something is not set up correctly.

image
image512×537 62.6 KB

image
image897×742 144 KB

here is the commands you requested

root@OpenWrt:~# tc -s qdisc show dev eth4
qdisc cake 8015: root refcnt 2 bandwidth 45Mbit besteffort triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 1164671088 bytes 4723712 pkt (dropped 731, overlimits 4163811 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1504304b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           42 /    1514
 min/max overhead-adjusted size:       42 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         45Mbit
  target            5ms
  interval        100ms
  pk_delay        226us
  av_delay         10us
  sp_delay          1us
  backlog            0b
  pkts          4724443
  bytes      1165729043
  way_inds       301215
  way_miss        80954
  way_cols            0
  drops             731
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len         30280
  quantum          1373

qdisc ingress ffff: parent ffff:fff1 ----------------
 Sent 17787768333 bytes 18031518 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
root@OpenWrt:~#
root@OpenWrt:~# tc -s qdisc show dev ifb4eth4
qdisc cake 8016: root refcnt 2 bandwidth 750Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms raw overhead 0
 Sent 18252732329 bytes 18055197 pkt (dropped 1894, overlimits 13989415 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 15170998b of 15140Kb
 capacity estimate: 750Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh        750Mbit
  target            5ms
  interval        100ms
  pk_delay         25us
  av_delay          7us
  sp_delay          2us
  backlog            0b
  pkts         18057091
  bytes     18255581411
  way_inds       159663
  way_miss        79902
  way_cols            0
  drops            1894
  marks               0
  ack_drop            0
  sp_flows            3
  bk_flows            1
  un_flows            0
  max_len         66616
  quantum          1514

root@OpenWrt:~#
12

Results of the MTR

image
image1531×588 185 KB

13

I am getting a nano pi4s this Friday (hopefully). I will be testing that too...hopefully is not my ISP and Just a bad set up.

14

Ok so I am still stomped about this, i am running a PC with a i7, 16gb, 180gb HDD, Intel 340 4 port GB card...and Cake still doesn't work.

I am getting between 30 ms to 65 ms using Fq Codel simple QOS, but as soon as change to Cake, it goes up to 280 ms. the only reason this is killing me is because everyone says Cake is way better and i should see better results, i am so confused.

15

It seems the only way i can get cake to work is to set my downloads speed to 150000 or below. Not sure if this worth it, as my speed test show i can get up to 800000 downloads. is this normal?

cake2
cake21015×631 56.9 KB

16

That seems on the low end for an i7.... if you install and run powertop concurrently to a speedtest, what frequency do the cores run at? And are you running OpenWrt on pure metal or inside a VM? (You might have answered all of this before)

17

Running pure metal. not sure if netdata would be the same as powertop

but as you can see none of the 8 cores moved...when running fast.com.

image
image1254×620 98.8 KB

18

No netdata and powertop cater to different use cases. I wonder whether the issue might be CPU latency, that is whether cake gets access to the CPU sufficiently fast when it needs to. The challenge here is that cake is not a continous load that is easy to anticipate but burst of high demand with periods of absolute quit in between. Some ARM SoCs struggle with cake for the same reason, there sometimes it hekps to a) spread qake instances to separate CPUs and effectively disable piwersaving features so the CPU stays in its highest sustainable frequency.

19

Way far away from my understanding of SQM, how would I tell Cake to do its instances in different cpu's? :slight_smile:

20

The typical approaches would be to either use irqbalance, packet_steering or both.