[SOLVED] SQM on Shuttle DS77U - cannot get it working

Installing and Using OpenWrt
1

(This topic relates to 18.06-rc1)

My old router, a Linksys EA8500 (ipq806x), was unable to shape 100Mbit, and topped out around 92 with 100% irq load on 1 core.

So, out with the old and in with the new, and I bought myself a Shuttle DS77U. It has the Celeron 3865U. It has dual intel gigabit, and I'm just using the simple WAN in one port, LAN in the other to a dumb switch.

I cannot for the life of me get SQM to work properly. The ingress and egress settings don't seem to have any effect and the dslreports speedtest shows full bufferbloat ~500ms.

ifconfig:

br-lan    Link encap:Ethernet  HWaddr 80:EE:73:D2:B9:55  
          inet addr:192.168.1.1  Bcast:192.168.1.255  Mask:255.255.255.0
          inet6 addr: fe80::82ee:73ff:fed2:b955/64 Scope:Link
          inet6 addr: 2600:6c44:527f:e71f::1/64 Scope:Global
          inet6 addr: fd8b:869f:3c13::1/60 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1298275 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2059444 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:164342284 (156.7 MiB)  TX bytes:2429098283 (2.2 GiB)

eth0      Link encap:Ethernet  HWaddr 80:EE:73:D2:B9:55  
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1309532 errors:0 dropped:19 overruns:0 frame:0
          TX packets:2589897 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:183226142 (174.7 MiB)  TX bytes:2464354389 (2.2 GiB)
          Memory:df000000-df01ffff 

eth1      Link encap:Ethernet  HWaddr 80:EE:73:D2:B9:54  
          inet addr:71.xx.xx.xxx  Bcast:71.xx.xx.255  Mask:255.255.254.0
          inet6 addr: fe80::82ee:73ff:fed2:b954/64 Scope:Link
          inet6 addr: 2600:6c44:7004:100:xxxx:xxxx:xxxx:xxxx/128 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2181936 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1244996 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:2270370957 (2.1 GiB)  TX bytes:208018850 (198.3 MiB)
          Interrupt:16 Memory:df100000-df120000 

ifb4eth1  Link encap:Ethernet  HWaddr 6A:7C:2E:73:95:EE  
          inet6 addr: fe80::687c:2eff:fe73:95ee/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:28768 errors:0 dropped:0 overruns:0 frame:0
          TX packets:28768 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:32 
          RX bytes:33763959 (32.1 MiB)  TX bytes:33763959 (32.1 MiB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:5632 errors:0 dropped:0 overruns:0 frame:0
          TX packets:5632 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:419158 (409.3 KiB)  TX bytes:419158 (409.3 KiB)

tun0      Link encap:UNSPEC  HWaddr 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.8.0.1  P-t-P:10.8.0.2  Mask:255.255.255.255
          inet6 addr: fe80::65e9:af65:a1ce:c360/64 Scope:Link
          UP POINTOPOINT RUNNING NOARP MULTICAST  MTU:1500  Metric:1
          RX packets:28927 errors:0 dropped:0 overruns:0 frame:0
          TX packets:34122 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100 
          RX bytes:3849635 (3.6 MiB)  TX bytes:31900477 (30.4 MiB)

/etc/config/sqm:


config queue 'eth1'
	option ingress_ecn 'ECN'
	option itarget 'auto'
	option etarget 'auto'
	option upload '10800'
	option debug_logging '0'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option qdisc_advanced '1'
	option squash_dscp '1'
	option squash_ingress '1'
	option egress_ecn 'NOECN'
	option qdisc_really_really_advanced '1'
	option iqdisc_opts 'mpu 64'
	option eqdisc_opts 'mpu 64'
	option linklayer 'ethernet'
	option overhead '18'
	option linklayer_advanced '1'
	option tcMTU '2047'
	option tcTSIZE '128'
	option tcMPU '0'
	option linklayer_adaptation_mechanism 'cake'
	option interface 'eth1'
	option enabled '1'
	option verbosity '8'
	option download '108300'

logread | grep SQM:

Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: Stopping SQM on eth1
Thu Jul 12 14:53:57 2018 user.notice SQM: Starting SQM script: piece_of_cake.qos on eth1, in: 108300 Kbps, out: 10800 Kbps
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: Starting SQM script: piece_of_cake.qos on eth1, in: 108300 Kbps, out: 10800 Kbps
Thu Jul 12 14:53:57 2018 user.notice SQM: QDISC cake is useable.
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: QDISC cake is useable.
Thu Jul 12 14:53:57 2018 user.notice SQM: Starting piece_of_cake.qos
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: Starting piece_of_cake.qos
Thu Jul 12 14:53:57 2018 user.notice SQM: ifb associated with interface eth1:
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: ifb associated with interface eth1:
Thu Jul 12 14:53:57 2018 user.notice SQM: Currently no ifb is associated with eth1, this is normal during starting of the sqm system.
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: Currently no ifb is associated with eth1, this is normal during starting of the sqm system.
Thu Jul 12 14:53:57 2018 user.notice SQM: egress
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: egress
Thu Jul 12 14:53:57 2018 user.notice SQM: cake link layer adjustments:  overhead 18 mpu 0
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: cake link layer adjustments:  overhead 18 mpu 0
Thu Jul 12 14:53:57 2018 user.notice SQM: egress shaping activated
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: egress shaping activated
Thu Jul 12 14:53:57 2018 user.notice SQM: QDISC ingress is useable.
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: QDISC ingress is useable.
Thu Jul 12 14:53:57 2018 user.notice SQM: ingress
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: ingress
Thu Jul 12 14:53:57 2018 user.notice SQM: cake link layer adjustments:  overhead 18 mpu 0
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: cake link layer adjustments:  overhead 18 mpu 0
Thu Jul 12 14:53:57 2018 user.notice SQM: ingress shaping activated
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: ingress shaping activated
Thu Jul 12 14:53:57 2018 user.notice SQM: piece_of_cake.qos was started on eth1 successfully
Thu Jul 12 14:53:57 2018 daemon.err uhttpd[2118]: SQM: piece_of_cake.qos was started on eth1 successfully

tc -d qdisc:

qdisc noqueue 0: dev lo root refcnt 2 
qdisc mq 0: dev eth0 root 
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc cake 8028: dev eth1 root refcnt 2 besteffort flowblind rtt 5.0ms raw overhead 18 mpu 1 
qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
qdisc noqueue 0: dev br-lan root refcnt 2 
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
qdisc cake 8029: dev ifb4eth1 root refcnt 2 besteffort flowblind wash rtt 5.0ms raw overhead 18 mpu 1

Sorry for the information overload. It occurs to me that this could also be an issue with 18.06-rc1, as the upgrade to that and the new pc came at the same time, but perhaps it's just an issue with the interface names or something.

2

The output of "tc -s qdisc" would also be helpful.

That said,

and

look quite wrong and not in line with /etc/config/sqm...

For one the default rtt should be 100.0ms not 5.0ms, and you explicitly requested MPU 64 (and on newer sqm-scripts this might be overruled by the tcMPU, but that would be zero, the "mpu 1" is completely bonkers).

I wonder whether tc, sqm-scripts and sch_cake in 18.06-rc1 have not been updated when the versions in master had been?
@ldir, since you know so much more about this matters than I do, do you have any idea why 18.06 might show these issues?

Side note, newer sqm-scripts will actually take the content of tcMPU seriously, allowing you to skip the [e|i]qdisc_opts

3

Here is the output of tc -s qdisc after running a speed test:

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 2979254230 bytes 3027730 pkt (dropped 0, overlimits 0 requeues 3116) 
 backlog 0b 0p requeues 3116
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 12668451 bytes 26143 pkt (dropped 0, overlimits 0 requeues 125) 
 backlog 0b 0p requeues 125
  maxpacket 1942 drop_overlimit 0 new_flow_count 51 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 2966585779 bytes 3001587 pkt (dropped 0, overlimits 0 requeues 2991) 
 backlog 0b 0p requeues 2991
  maxpacket 9058 drop_overlimit 0 new_flow_count 1339 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc cake 8028: dev eth1 root refcnt 2 besteffort flowblind rtt 5.0ms raw overhead 18 mpu 1 
 Sent 134697508 bytes 312365 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 14b of 39424b
 capacity estimate: 6755Tbit
 min/max network layer size:         1514 /      42
 min/max overhead-adjusted size:     1514 /   65708
 average network hdr offset:           42


qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 539181666 bytes 459435 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 86636723 bytes 77854 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 40 drop_overlimit 0 new_flow_count 58 ecn_mark 0
  new_flows_len 1 old_flows_len 0
qdisc cake 8029: dev ifb4eth1 root refcnt 2 besteffort flowblind wash rtt 5.0ms raw overhead 18 mpu 1 
 Sent 550317412 bytes 459435 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
 memory used: 14b of 23296b
 capacity estimate: 6755Tbit
 min/max network layer size:         1514 /      60
 min/max overhead-adjusted size:     1514 /   65708
 average network hdr offset:           60
4

Mmmhh, this output also looks quite wrong, I wonder whether this might be caused by incompatible TC and sch-cake. Could you, for testing purposes, switch to simple.qos/fq_codel. And redo the desreports speedtests please?

5

I switched to simple.qos/fq_codel and set ingress/egress to 5000. This did actually limit the speed this time. I see this is the same result that was observed in the other thread.

Here is the tc -s output after running the test:

qdisc noqueue 0: dev lo root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root 
 Sent 1591290601 bytes 1820162 pkt (dropped 0, overlimits 0 requeues 1980) 
 backlog 0b 0p requeues 1980
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 5252473 bytes 13712 pkt (dropped 0, overlimits 0 requeues 59) 
 backlog 0b 0p requeues 59
  maxpacket 549 drop_overlimit 0 new_flow_count 27 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 1586038128 bytes 1806450 pkt (dropped 0, overlimits 0 requeues 1921) 
 backlog 0b 0p requeues 1921
  maxpacket 25738 drop_overlimit 0 new_flow_count 900 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev eth1 root refcnt 2 r2q 10 default 18 direct_packets_stat 0 direct_qlen 1000
 Sent 8139027 bytes 13348 pkt (dropped 0, overlimits 10086 requeues 0) 
 backlog 0b 0p requeues 0
qdisc ingress ffff: dev eth1 parent ffff:fff1 ---------------- 
 Sent 9231635 bytes 11911 pkt (dropped 752, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 32Mb ecn 
 Sent 8668907 bytes 11767 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
  maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc htb 1: dev ifb4eth1 root refcnt 2 r2q 10 default 16 direct_packets_stat 0 direct_qlen 32
 Sent 8255365 bytes 11092 pkt (dropped 752, overlimits 11093 requeues 0) 
 backlog 0b 0p requeues 0
6

Here is some package info. This is kernel 4.14 on x86_64.

kmod-sched-cake - 4.14.50+2018-05-15-779e91a9-1 - Common Applications Kept Enhanced fq_codel/blue derived shaper

tc - 4.16.0-3 - Traffic control utility
7

I switched to master and SQM is working now.