SQM startup debugging discussion

@pensioner600 let's continue our discussion about sqm from here:

in this thread.

To understand where you are I would like you to switch back to sqm-sxripts and post the output of the following commands for the manually started sqm instance:

  1. ifstatus wan | grep device
  2. cat /etc/config/sqm
  3. tc -s qdisc

This should give me an understanding how things look when they work and then we can take it from there...

Was there any insight?
Just yesterday I realized that my minimal sqm comfig does not work anymore or where not active at all all the time. I'm still on 23.05 so should upgrade soon anyway but I would have thought that there is not much progress with sqm as it's rather old and stable.

How do I debug the setup of sqm rules?

So far I got no feed back from the poster. On my turris omnia I had similar problem earlier on, but "solved" them in a way to ugly to share (I take the net hotplug events and just create the missing iface hotplug events that sqm expects... but that solved the issue for me locally, but is not a proper solution). I might find some time over the next two weeks to look deeper into this... (One challenge is my omnia was OpenWrt22 based until a few weeks ago when the base was updated to OpenWrt24 based, so only now can I expect my omnia to be actually somewhat reflective of what others are using...)

But maybe, since you have issues as well, you might be willing to help me debug this?

This is easy:

/etc/init.d/sqm stop # make sure nothing runs
/etc/init.d sqm start
tc -s qdisc
tc -d qdisc

but if you are facing the hotplug issue this will work just fine and it is only after hotplug you will run into issues, and likely only on pppoe-wan, as that interface can go away completely and unless we catch the hotplug event for pppoe-wan existing/becomung UP again sqm on pppoe-wan will not survive e.g. a pppoe reconnect.

root@cpe:~# /etc/init.d/sqm status
inactive
root@cpe:~# /etc/init.d/sqm start
root@cpe:~# /etc/init.d/sqm status
active with no instances

root@cpe:~# cat /etc/config/sqm

config queue 'pppoe-wan'
    option  enabled                         '1'
    option  interface                       'pppoe-wan'
    option  linklayer                       'ethernet'
    option  download                        '105000'
    option  upload                           '47000'
    option  qdisc                           'cake'
    option  script                          'layer_cake.qos'


root@cpe:~# tc -d qdisc show dev pppoe-wan
qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64

root@cpe:~# tc -s qdisc show dev pppoe-wan
qdisc fq_codel 0: root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64
 Sent 16924274110 bytes 92046612 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 21900 drop_overlimit 0 new_flow_count 1917086 ecn_mark 0
  new_flows_len 0 old_flows_len 0

1 Like

This is not exactly what I asked for, but if you want to keep your network internals internal, by all means do so, but please provide the output for both pppoe-wan and ifb4pppoe-wan.

But clearly that /etc/init.d/sqm start invocation got nowhere. Could you run:

/etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start

and the copy and paste the command output, please?

Pardon me, but

  1. Main issue is: sqm is not running; I concluded/expected: no applied tc rules.
  2. My half part understanding of sqm is, that I want to shape (regarding on my config) only traffic on the pppoe-wan interface. My wrong assumption was, that we want to see / confirm no special rules are in place.
  3. Everything was just qdisc noqueue and qdisc fq_codel, but no sight of cake. But I can for sure provide the boring full dump.

Sure.


BusyBox v1.36.1 (2025-01-08 18:59:28 UTC) built-in shell (ash)

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 -----------------------------------------------------
 OpenWrt 23.05.5, r24106-10cc5fcd00
 -----------------------------------------------------
root@cpe:~#
root@cpe:~# { /etc/init.d/sqm stop ; SQM_DEBUG=1 SQM_VERBOSITY_MAX=8 /etc/init.d/sqm start; } 2>&1 | tee "/tmp/forum-$(date).log"
/usr/lib/sqm/run.sh: line 57: can't create : nonexistent directory
root@cpe:~#
root@cpe:~# find /var/run/sqm/available_qdiscs/
/var/run/sqm/available_qdiscs/
/var/run/sqm/available_qdiscs/cake
/var/run/sqm/available_qdiscs/fq_codel

I have tested with this extended / stating all defaults config:

root@cpe:~# cat /etc/config/sqm

config queue 'pppoe-wan'
    option  enabled                         '1'
    option  interface                       'pppoe-wan'
    option  linklayer                       'ethernet'
    option  download                        '105000'
    option  upload                           '47000'
    option  qdisc                           'cake'
    option  script                          'layer_cake.qos'
    option  tcMPU                           '64'
    option  ilimit                          ''
    option  elimit                          ''
    option  itarget                         'default'
    option  etarget                         'default'
    option  ingress_ecn                     'ECN'
    option  egress_ecn                      'ECN'
    option  target                          'default'
    option  squash_dscp                     '1'
    option  squash_ingress                  '1'


root@cpe:~#

:melting_face:

root@cpe:~# uci show sqm
uci: Parse error (invalid character in name field) at line 1, byte 25
root@cpe:~# vim /etc/config/sqm
E1187: Failed to source defaults.vim
Press ENTER or type command to continue
root@cpe:~# uci show sqm
sqm.pppoewan=queue
sqm.pppoewan.enabled='1'
sqm.pppoewan.interface='pppoe-wan'
sqm.pppoewan.linklayer='ethernet'
sqm.pppoewan.download='105000'
sqm.pppoewan.upload='47000'
sqm.pppoewan.qdisc='cake'
sqm.pppoewan.script='layer_cake.qos'
sqm.pppoewan.tcMPU='64'
sqm.pppoewan.itarget='default'
sqm.pppoewan.etarget='default'
sqm.pppoewan.ingress_ecn='ECN'
sqm.pppoewan.egress_ecn='ECN'
sqm.pppoewan.target='default'
sqm.pppoewan.squash_dscp='1'
sqm.pppoewan.squash_ingress='1'

Spot the difference. :expressionless_face:

1 Like

Here we go:

SQM: ifb associated with interface pppoe-wan: ifb4pppoe-wan
SQM: Stopping SQM on pppoe-wan
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc del dev pppoe-wan ingress
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc del dev pppoe-wan root
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete dev ifb4pppoe-wan type ifb
SQM: /usr/lib/sqm/stop-sqm: ifb4pppoe-wan interface deleted
/usr/lib/sqm/run.sh: line 57: can't create : nonexistent directory
SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 105000 Kbps, out: 47000 Kbps
SQM: fn_exists: function candidate name: sqm_start
SQM: fn_exists: TYPE_OUTPUT: sqm_start: not found
SQM: fn_exists: return value: 1
SQM: Using generic sqm_start_default function.
SQM: fn_exists: function candidate name: sqm_prepare_script
SQM: fn_exists: TYPE_OUTPUT: sqm_prepare_script is a function
SQM: fn_exists: return value: 0
SQM: sqm_start_default: starting sqm_prepare_script
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_cbc8b type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_cbc8b root cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_cbc8b down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_cbc8b type ifb
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_1c31b type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_1c31b root cake
SQM: QDISC cake is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_1c31b down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_1c31b type ifb
SQM: sqm_start_default: Starting layer_cake.qos
SQM: ifb associated with interface pppoe-wan: 
SQM: Currently no ifb is associated with pppoe-wan, this is normal during starting of the sqm system.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name ifb4pppoe-wan type ifb
SQM: fn_exists: function candidate name: egress
SQM: fn_exists: TYPE_OUTPUT: egress is a function
SQM: fn_exists: return value: 0
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev pppoe-wan root
SQM: cmd_wrapper: tc: LAST ERROR: Error: Cannot delete qdisc with handle of zero.
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cake link layer adjustments:  overhead 0 mpu 64
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev pppoe-wan root cake bandwidth 47000kbit overhead 0 mpu 64 diffserv3
SQM: sqm_start_default: egress shaping activated
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link add name SQM_IFB_97f95 type ifb
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc replace dev SQM_IFB_97f95 ingress
SQM: QDISC ingress is useable.
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev SQM_IFB_97f95 down
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link delete SQM_IFB_97f95 type ifb
SQM: fn_exists: function candidate name: ingress
SQM: fn_exists: TYPE_OUTPUT: ingress is a function
SQM: fn_exists: return value: 0
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev pppoe-wan handle ffff: ingress
SQM: cmd_wrapper: tc: LAST ERROR: Error: Invalid handle.
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev pppoe-wan handle ffff: ingress
SQM: cmd_wrapper: tc: invocation silenced by request, FAILURE either expected or acceptable.
SQM: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev ifb4pppoe-wan root
SQM: cmd_wrapper: tc: LAST ERROR: Error: Cannot delete qdisc with handle of zero.
SQM: LLA: default link layer adjustment method for cake is cake
SQM: cake link layer adjustments:  overhead 0 mpu 64
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc qdisc add dev ifb4pppoe-wan root cake bandwidth 105000kbit overhead 0 mpu 64 diffserv3 besteffort wash
SQM: cmd_wrapper: ip: SUCCESS: /sbin/ip link set dev ifb4pppoe-wan up
SQM: cmd_wrapper: tc: SUCCESS: /sbin/tc filter add dev pppoe-wan parent ffff: protocol all prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb4pppoe-wan
SQM: sqm_start_default: ingress shaping activated
SQM: layer_cake.qos was started on pppoe-wan successfully
qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 1844748570 bytes 1357914 pkt (dropped 0, overlimits 0 requeues 321) 
 backlog 0b 0p requeues 321
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 1229937606 bytes 917285 pkt (dropped 0, overlimits 0 requeues 228) 
 backlog 0b 0p requeues 228
  maxpacket 1414 drop_overlimit 0 new_flow_count 30 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 614810964 bytes 440629 pkt (dropped 0, overlimits 0 requeues 93) 
 backlog 0b 0p requeues 93
  maxpacket 1414 drop_overlimit 0 new_flow_count 14 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth3 root 
 Sent 31164366 bytes 292401 pkt (dropped 0, overlimits 0 requeues 516) 
 backlog 0b 0p requeues 516
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 13152190 bytes 123572 pkt (dropped 0, overlimits 0 requeues 306) 
 backlog 0b 0p requeues 306
  maxpacket 1502 drop_overlimit 0 new_flow_count 284 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 18012176 bytes 168829 pkt (dropped 0, overlimits 0 requeues 210) 
 backlog 0b 0p requeues 210
  maxpacket 1514 drop_overlimit 0 new_flow_count 165 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev dummy0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.16 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.17 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.24 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.49 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.56 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.64 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.65 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.71 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.76 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.77 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br1-vlan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br1-vlan.4094 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth3.7 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.16 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3035_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_3 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.17 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3914_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1771_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.24 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3035_2 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_autopeer root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wg15 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_2 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.49 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wg0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.56 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.64 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.65 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.71 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.76 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.77 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 47Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 0b of 4Mb
 capacity estimate: 47Mbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       2937Kbit       47Mbit    11750Kbit
  target         6.18ms          5ms          5ms
  interval        101ms        100ms        100ms
  pk_delay          0us          0us          0us
  av_delay          0us          0us          0us
  sp_delay          0us          0us          0us
  backlog            0b           0b           0b
  pkts                0            0            0
  bytes               0            0            0
  way_inds            0            0            0
  way_miss            0            0            0
  way_cols            0            0            0
  drops               0            0            0
  marks               0            0            0
  ack_drop            0            0            0
  sp_flows            0            0            0
  bk_flows            0            0            0
  un_flows            0            0            0
  max_len             0            0            0
  quantum           300         1434          358

qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ---------------- 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 105Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 0b of 5250000b
 capacity estimate: 105Mbit
 min/max network layer size:        65535 /       0
 min/max overhead-adjusted size:    65535 /       0
 average network hdr offset:            0

                  Tin 0
  thresh        105Mbit
  target            5ms
  interval        100ms
  pk_delay          0us
  av_delay          0us
  sp_delay          0us
  backlog            0b
  pkts                0
  bytes               0
  way_inds            0
  way_miss            0
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            0
  un_flows            0
  max_len             0
  quantum          1514

qdisc noqueue 0: dev lo root refcnt 2 
qdisc mq 0: dev eth0 root 
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc mq 0: dev eth1 root 
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc mq 0: dev eth3 root 
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc noqueue 0: dev dummy0 root refcnt 2 
qdisc noqueue 0: dev br-vlan root refcnt 2 
qdisc noqueue 0: dev br-vlan.16 root refcnt 2 
qdisc noqueue 0: dev br-vlan.17 root refcnt 2 
qdisc noqueue 0: dev br-vlan.24 root refcnt 2 
qdisc noqueue 0: dev br-vlan.49 root refcnt 2 
qdisc noqueue 0: dev br-vlan.56 root refcnt 2 
qdisc noqueue 0: dev br-vlan.64 root refcnt 2 
qdisc noqueue 0: dev br-vlan.65 root refcnt 2 
qdisc noqueue 0: dev br-vlan.71 root refcnt 2 
qdisc noqueue 0: dev br-vlan.76 root refcnt 2 
qdisc noqueue 0: dev br-vlan.77 root refcnt 2 
qdisc noqueue 0: dev br1-vlan root refcnt 2 
qdisc noqueue 0: dev br1-vlan.4094 root refcnt 2 
qdisc noqueue 0: dev eth3.7 root refcnt 2 
qdisc noqueue 0: dev bat0 root refcnt 2 
qdisc noqueue 0: dev bat0.16 root refcnt 2 
qdisc noqueue 0: dev dn42_3035_1 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_3 root refcnt 2 
qdisc noqueue 0: dev bat0.17 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_1 root refcnt 2 
qdisc noqueue 0: dev dn42_3914_1 root refcnt 2 
qdisc noqueue 0: dev dn42_1771_1 root refcnt 2 
qdisc noqueue 0: dev bat0.24 root refcnt 2 
qdisc noqueue 0: dev dn42_3035_2 root refcnt 2 
qdisc noqueue 0: dev dn42_autopeer root refcnt 2 
qdisc noqueue 0: dev wg15 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_2 root refcnt 2 
qdisc noqueue 0: dev bat0.49 root refcnt 2 
qdisc noqueue 0: dev wg0 root refcnt 2 
qdisc noqueue 0: dev bat0.56 root refcnt 2 
qdisc noqueue 0: dev bat0.64 root refcnt 2 
qdisc noqueue 0: dev bat0.65 root refcnt 2 
qdisc noqueue 0: dev bat0.71 root refcnt 2 
qdisc noqueue 0: dev bat0.76 root refcnt 2 
qdisc noqueue 0: dev bat0.77 root refcnt 2 
qdisc cake 8009: dev pppoe-wan root refcnt 2 bandwidth 47Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ---------------- 
qdisc cake 800a: dev ifb4pppoe-wan root refcnt 2 bandwidth 105Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 


Edit 23:00

I slightly reduced download and upload speeds; video calls seams to work; youtube has struggles; parallel radio stream is not impacted; and 100G /dev/zero with wget all the time goes up and down, and after the reduce to 100/45 I get waveform bufferbloat test of A. So it seams to work now.

1 Like

Now, the interesting question is, what happens on a pppoe-wan reconnect (or a cold boot for that matter) will sqm start/restart automatically in those cases, or will it stay disabled?

I can test this now...

1 Like

Is this a german VDSL2 link, by any chance?

In which case you should likely add the required overhead... overhead 0 is always incorrect...

As the config/sqm was now correct, I already assumed its reboot safe:

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 2133140 bytes 3499 pkt (dropped 0, overlimits 0 requeues 4) 
 backlog 0b 0p requeues 4
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 328711 bytes 982 pkt (dropped 0, overlimits 0 requeues 2) 
 backlog 0b 0p requeues 2
  maxpacket 110 drop_overlimit 0 new_flow_count 3 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 1804429 bytes 2517 pkt (dropped 0, overlimits 0 requeues 2) 
 backlog 0b 0p requeues 2
  maxpacket 142 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth3 root 
 Sent 1099768 bytes 3176 pkt (dropped 0, overlimits 0 requeues 1) 
 backlog 0b 0p requeues 1
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 491359 bytes 1237 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
 Sent 608409 bytes 1939 pkt (dropped 0, overlimits 0 requeues 1) 
 backlog 0b 0p requeues 1
  maxpacket 94 drop_overlimit 0 new_flow_count 1 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev dummy0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.16 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.17 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.24 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.49 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.56 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.64 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.65 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.71 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.76 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-vlan.77 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br1-vlan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br1-vlan.4094 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth3.7 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_2 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.16 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3035_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.17 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1588_3 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_1771_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3914_1 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_3035_2 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.24 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev dn42_autopeer root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wg15 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wg0 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.49 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.56 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.64 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.65 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.71 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.76 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev bat0.77 root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 800d: dev pppoe-wan root refcnt 2 bandwidth 45Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
 Sent 980551 bytes 2800 pkt (dropped 1, overlimits 977 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 41428b of 4Mb
 capacity estimate: 45Mbit
 min/max network layer size:           40 /    1492
 min/max overhead-adjusted size:       64 /    1492
 average network hdr offset:            0

                   Bulk  Best Effort        Voice
  thresh       2812Kbit       45Mbit    11250Kbit
  target         6.46ms          5ms          5ms
  interval        101ms        100ms        100ms
  pk_delay          0us         61us          6us
  av_delay          0us         12us          0us
  sp_delay          0us          1us          0us
  backlog            0b           0b           0b
  pkts                0         2768           33
  bytes               0       977384         4647
  way_inds            0           31            0
  way_miss            0          313            9
  way_cols            0            0            0
  drops               0            1            0
  marks               0            0            0
  ack_drop            0            0            0
  sp_flows            0            0            1
  bk_flows            0            1            0
  un_flows            0            0            0
  max_len             0         2895          237
  quantum           300         1373          343

qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ---------------- 
 Sent 2403184 bytes 3353 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc cake 800e: dev ifb4pppoe-wan root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
 Sent 2403184 bytes 3353 pkt (dropped 0, overlimits 2193 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 20736b of 5000000b
 capacity estimate: 100Mbit
 min/max network layer size:           35 /    1492
 min/max overhead-adjusted size:       64 /    1492
 average network hdr offset:            0

                  Tin 0
  thresh        100Mbit
  target            5ms
  interval        100ms
  pk_delay        147us
  av_delay         45us
  sp_delay          5us
  backlog            0b
  pkts             3353
  bytes         2403184
  way_inds           47
  way_miss          325
  way_cols            0
  drops               0
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1492
  quantum          1514

qdisc noqueue 0: dev lo root refcnt 2 
qdisc mq 0: dev eth0 root 
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc mq 0: dev eth1 root 
qdisc fq_codel 0: dev eth1 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc mq 0: dev eth3 root 
qdisc fq_codel 0: dev eth3 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc fq_codel 0: dev eth3 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 32Mb ecn drop_batch 64 
qdisc noqueue 0: dev dummy0 root refcnt 2 
qdisc noqueue 0: dev br-vlan root refcnt 2 
qdisc noqueue 0: dev br-vlan.16 root refcnt 2 
qdisc noqueue 0: dev br-vlan.17 root refcnt 2 
qdisc noqueue 0: dev br-vlan.24 root refcnt 2 
qdisc noqueue 0: dev br-vlan.49 root refcnt 2 
qdisc noqueue 0: dev br-vlan.56 root refcnt 2 
qdisc noqueue 0: dev br-vlan.64 root refcnt 2 
qdisc noqueue 0: dev br-vlan.65 root refcnt 2 
qdisc noqueue 0: dev br-vlan.71 root refcnt 2 
qdisc noqueue 0: dev br-vlan.76 root refcnt 2 
qdisc noqueue 0: dev br-vlan.77 root refcnt 2 
qdisc noqueue 0: dev br1-vlan root refcnt 2 
qdisc noqueue 0: dev br1-vlan.4094 root refcnt 2 
qdisc noqueue 0: dev bat0 root refcnt 2 
qdisc noqueue 0: dev eth3.7 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_2 root refcnt 2 
qdisc noqueue 0: dev bat0.16 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_1 root refcnt 2 
qdisc noqueue 0: dev dn42_3035_1 root refcnt 2 
qdisc noqueue 0: dev bat0.17 root refcnt 2 
qdisc noqueue 0: dev dn42_1588_3 root refcnt 2 
qdisc noqueue 0: dev dn42_1771_1 root refcnt 2 
qdisc noqueue 0: dev dn42_3914_1 root refcnt 2 
qdisc noqueue 0: dev dn42_3035_2 root refcnt 2 
qdisc noqueue 0: dev bat0.24 root refcnt 2 
qdisc noqueue 0: dev dn42_autopeer root refcnt 2 
qdisc noqueue 0: dev wg15 root refcnt 2 
qdisc noqueue 0: dev wg0 root refcnt 2 
qdisc noqueue 0: dev bat0.49 root refcnt 2 
qdisc noqueue 0: dev bat0.56 root refcnt 2 
qdisc noqueue 0: dev bat0.64 root refcnt 2 
qdisc noqueue 0: dev bat0.65 root refcnt 2 
qdisc noqueue 0: dev bat0.71 root refcnt 2 
qdisc noqueue 0: dev bat0.76 root refcnt 2 
qdisc noqueue 0: dev bat0.77 root refcnt 2 
qdisc cake 800d: dev pppoe-wan root refcnt 2 bandwidth 45Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 
qdisc ingress ffff: dev pppoe-wan parent ffff:fff1 ---------------- 
qdisc cake 800e: dev ifb4pppoe-wan root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 64 

I will now test with physical carrier down, and carrier up event....

Correct. Its DTAG Magenta (M?) 100/50 and they deliver 112 and 50. And yes my original overhead reserve was too small. What recommended percentage was it again?

100/50? That means it is likely FTTH/GPON, which is a bit tricky... (the problem is we really need to set for the maximum of the true overhead and whatever Telekom configured in its traffic shaper...)
My best guess for PPPoE/VLAN7/ overhead on [G|XGS]-PON would be:
GPON Overhead (PPPoE)
#GPON-GEM: GEM = 5 Byte
COMMON ETHERNET: 4 Byte Frame Check Sequence (FCS) + 6 (dest MAC) + 6 (src MAC) + 2 (ethertype) = 18 byte
PPPoE: 8 Byte
VLAN: 4 Byte
TOTAL: 5+18+8+4 = 35

so 35 bytes on top of pppoe-wan. And MPU 64+5 = 69.

That depends on your MTU :wink: I prefer to look at per packet overhead as absolute numbers, not the least as that is how you need to configure them.

I hope that's all.
After I inserted the ethernet again into eth3, the pppoe re-connection lasted like 2 or 3 min.

Dec 23 19:19:08 cpe netifd: Interface 'wan' is now down
Dec 23 19:19:11 cpe netifd: Network device 'eth3' link is up
Dec 23 19:19:11 cpe netifd: VLAN 'eth3.7' link is up
Dec 23 19:19:11 cpe netifd: Interface 'wan' has link connectivity
Dec 23 19:19:11 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:19:26 cpe netifd: Interface 'wan' is now down
Dec 23 19:19:26 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:19:42 cpe netifd: Interface 'wan' is now down
Dec 23 19:19:42 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:19:57 cpe netifd: Interface 'wan' is now down
Dec 23 19:19:57 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:20:12 cpe netifd: Interface 'wan' is now down
Dec 23 19:20:12 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:20:27 cpe netifd: Interface 'wan' is now down
Dec 23 19:20:27 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:20:43 cpe netifd: Interface 'wan' is now down
Dec 23 19:20:43 cpe netifd: Interface 'wan' is setting up now
Dec 23 19:20:46 cpe netifd: Network device 'pppoe-wan' link is up
Dec 23 19:20:46 cpe netifd: Interface 'wan6' is enabled
Dec 23 19:20:46 cpe netifd: Network alias 'pppoe-wan' link is up
Dec 23 19:20:46 cpe netifd: Interface 'wan6' has link connectivity
Dec 23 19:20:46 cpe netifd: Interface 'wan6' is setting up now
Dec 23 19:20:46 cpe netifd: Interface 'wan' is now up
Dec 23 19:20:46 cpe netifd: Interface 'wan_6' is enabled
Dec 23 19:20:46 cpe netifd: Interface 'wan_6' has link connectivity
Dec 23 19:20:46 cpe netifd: Interface 'wan_6' is setting up now
Dec 23 19:20:46 cpe SQM: Stopping SQM on pppoe-wan
Dec 23 19:20:46 cpe SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev pppoe-wan ingress
Dec 23 19:20:46 cpe SQM: ERROR: cmd_wrapper: tc: LAST ERROR: Error: Cannot find specified qdisc on specified device.
Dec 23 19:20:46 cpe SQM: ERROR: cmd_wrapper: tc: FAILURE (2): /sbin/tc qdisc del dev pppoe-wan root
Dec 23 19:20:46 cpe SQM: ERROR: cmd_wrapper: tc: LAST ERROR: Error: Cannot delete qdisc with handle of zero.
Dec 23 19:20:47 cpe netifd: Interface 'wan6' is now up
Dec 23 19:20:47 cpe SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 100000 Kbps, out: 45000 Kbps
Dec 23 19:20:47 cpe SQM: layer_cake.qos was started on pppoe-wan successfully
Dec 23 19:20:47 cpe firewall: Reloading firewall due to ifup of wan (pppoe-wan)
Dec 23 19:20:47 cpe firewall: Reloading firewall due to ifupdate of wan (pppoe-wan)
Dec 23 19:20:48 cpe SQM: Stopping SQM on pppoe-wan
Dec 23 19:20:48 cpe SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 100000 Kbps, out: 45000 Kbps
Dec 23 19:20:49 cpe SQM: layer_cake.qos was started on pppoe-wan successfully
Dec 23 19:20:49 cpe firewall: Reloading firewall due to ifup of wan6 (pppoe-wan)
Dec 23 19:20:50 cpe netifd: Interface 'wan_6' is now up
Dec 23 19:20:50 cpe SQM: Stopping SQM on pppoe-wan
Dec 23 19:20:50 cpe SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 100000 Kbps, out: 45000 Kbps
Dec 23 19:20:50 cpe SQM: layer_cake.qos was started on pppoe-wan successfully
Dec 23 19:20:50 cpe firewall: Reloading firewall due to ifup of wan_6 (pppoe-wan)
Dec 23 19:20:53 cpe netifd: Interface 'wan_6' has lost the connection
Dec 23 19:20:53 cpe netifd: Interface 'wan_6' is now up
Dec 23 19:20:53 cpe SQM: Stopping SQM on pppoe-wan
Dec 23 19:20:53 cpe SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 100000 Kbps, out: 45000 Kbps
Dec 23 19:20:53 cpe SQM: layer_cake.qos was started on pppoe-wan successfully
Dec 23 19:20:53 cpe firewall: Reloading firewall due to ifup of wan_6 (pppoe-wan)

@moeller0 is this info enough?

root@cpe:~# /etc/init.d/sqm restart
SQM: Stopping SQM on pppoe-wan
SQM: Starting SQM script: layer_cake.qos on pppoe-wan, in: 100000 Kbps, out: 45000 Kbps
SQM: layer_cake.qos was started on pppoe-wan successfully
root@cpe:~# cat /etc/config/sqm
config queue 'pppoewan'
    option  enabled                         '1'
    option  interface                       'pppoe-wan'
    option  linklayer                       'ethernet'
    option  download                        '100000'
    option  upload                           '45000'
    option  qdisc                           'cake'
    option  script                          'layer_cake.qos'
    option  tcMPU                           '69'
    option  ilimit                          ''
    option  elimit                          ''
    option  itarget                         'default'
    option  etarget                         'default'
    option  ingress_ecn                     'ECN'
    option  egress_ecn                      'ECN'
    option  target                          'default'
    option  squash_dscp                     '1'
    option  squash_ingress                  '1'


root@cpe:~# date
Tue Dec 23 19:39:27 UTC 2025

I have tested bufferbloat again with 100G zero, radio and youtube, and video call, and still get an A.

root@cpe:~# tc -d qdisc show dev pppoe-wan; tc -s qdisc show dev ifb4pppoe-wan
qdisc cake 8031: root refcnt 2 bandwidth 45Mbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 69
qdisc ingress ffff: parent ffff:fff1 ----------------
qdisc cake 8032: root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 0 mpu 69
 Sent 1724993002 bytes 1246458 pkt (dropped 13526, overlimits 2336387 requeues 0)
 backlog 1280b 1p requeues 0
 memory used: 1451520b of 5000000b
 capacity estimate: 100Mbit
 min/max network layer size:           40 /    1492
 min/max overhead-adjusted size:       69 /    1492
 average network hdr offset:            0

                  Tin 0
  thresh        100Mbit
  target            5ms
  interval        100ms
  pk_delay        173us
  av_delay         64us
  sp_delay          3us
  backlog         1280b
  pkts          1259985
  bytes      1744205328
  way_inds         2426
  way_miss         1326
  way_cols            0
  drops           13526
  marks               0
  ack_drop            0
  sp_flows            1
  bk_flows            1
  un_flows            0
  max_len          1492
  quantum          1514

Try:

config queue 'eth1'
        option ingress_ecn 'ECN'
        option egress_ecn 'ECN'
        option itarget 'auto'
        option etarget 'auto'
        option verbosity '5'
        option qdisc 'cake'
        option qdisc_advanced '1'
        option squash_dscp '1'
        option squash_ingress '1'
        option qdisc_really_really_advanced '1'
        option linklayer 'ethernet'
        option linklayer_advanced '1'
        option tcMTU '2047'
        option tcTSIZE '128'
        option linklayer_adaptation_mechanism 'default'
        option debug_logging '1'
        option tcMPU '69'
        option enabled '1'
        option interface 'pppoe-wan'
        option upload '45000'
        option script 'layer_cake.qos'
        option overhead '35'
        option iqdisc_opts 'dual-dsthost nat ingress memlimit 32mb'
        option eqdisc_opts 'dual-srchost nat memlimit 32mb'
        option download '100000'

That adds option overhead '35' to set the correct over head and:
option iqdisc_opts 'dual-dsthost nat ingress memlimit 32mb' to enable NAT lookup and per-internal-IP fairness and it will increase the worst case buffering to 32 MB (almost never needed, but sometimes the default 4 MB in OpenWrt are a bit tight)
option eqdisc_opts 'dual-srchost nat memlimit 32mb' same for egress.

This looks like hotplug just works on your link, so good! However I still need to wait for feed-back/information from @pensioner600 ...

I have realized I've hijacked the thread already, so I've removed the solved mark :sweat_smile:

1 Like

Oh, I am happy, that you managed to solve your problem and that I could give some, hopefully useful, pointers for how to improve your config :wink: I am doubtful that pensioner will answer in this thread, so this achieved at least something positive :wink:

1 Like