Also can you post the output of:
nft list ruleset
Your "download" qdisc sees no data at all... can you also post the output of:
tc -s qdisc
please?
1 Like
@Lynx
Also can you post the output of:
nft list ruleset
table inet cake-qos-simple {
map rules_proto_dport {
type inet_proto . inet_service : verdict
elements = { tcp . 53 : goto dscp_set_voice,
udp . 53 : goto dscp_set_voice }
}chain hook-output { type filter hook output priority filter; policy accept; oifname "wan" goto classify-and-store-dscp } chain hook-forward { type filter hook forward priority filter; policy accept; iifname "br-lan" goto classify-and-store-dscp } chain classify-and-store-dscp { jump classify-dscp jump store-dscp-in-conntrack } chain classify-dscp { meta l4proto . th dport vmap @rules_proto_dport } chain dscp_set_bulk { ip dscp set cs1 } chain dscp_set_besteffort { ip dscp set cs0 } chain dscp_set_video { ip dscp set cs2 } chain dscp_set_voice { ip dscp set cs4 } chain store-dscp-in-conntrack { ip version 4 ct mark set ip dscp ct mark set ct mark | 0x00000080 }
cake-qos-simple configuration options:
ul_if=pppoe-wan # upload interface
dl_if="" # download interface override (normally left blank and IFB derived for $ul_if ingress)
cake_ul_rate_Mbps=17 # cake upload rate in Mbit/s
cake_dl_rate_Mbps=56 # cake download rate in Mbit/s
cake_ul_options="diffserv4 dual-srchost nonat wash no-ack-filter pppoe-ptm ether-vlan noatm"
cake_dl_options="diffserv4 dual-dsthost nonat nowash ingress no-ack-filter pppoe-ptm ether-vlan noatm"
# https://www.snbforums.com/threads/cake-and-wan-packet-overhead-vdsl2-pppoe-ptm-uk.83309/
# end of cake-qos-simple configuration options
Sure, Like i said it working fine on the upload but not the download.
root@R1:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 434018752628 bytes 383850522 pkt (dropped 0, overlimits 0 requeues 524)
backlog 0b 0p requeues 524
qdisc fq_codel 0: dev eth0 parent :4 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 109186254962 bytes 94014842 pkt (dropped 0, overlimits 0 requeues 132)
backlog 0b 0p requeues 132
maxpacket 6056 drop_overlimit 0 new_flow_count 743 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :3 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 114132724301 bytes 97386517 pkt (dropped 0, overlimits 0 requeues 147)
backlog 0b 0p requeues 147
maxpacket 4542 drop_overlimit 0 new_flow_count 454 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 95310919345 bytes 79165550 pkt (dropped 0, overlimits 0 requeues 101)
backlog 0b 0p requeues 101
maxpacket 4542 drop_overlimit 0 new_flow_count 385 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 115388854020 bytes 113283613 pkt (dropped 0, overlimits 0 requeues 144)
backlog 0b 0p requeues 144
maxpacket 4542 drop_overlimit 0 new_flow_count 636 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev lan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev lan2 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev lan3 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev lan4 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev phy0-ap0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev phy1-ap0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev dsl0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 20099242746 bytes 154344958 pkt (dropped 499, overlimits 0 requeues 527317)
backlog 0b 0p requeues 527317
maxpacket 1514 drop_overlimit 0 new_flow_count 67677 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev ifb-dns root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 5023748 bytes 33841 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev dsl0.101 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8023: dev pppoe-wan root refcnt 2 bandwidth 17Mbit diffserv4 dual-srchost nonat wash no-ack-filter split-gso rtt 100ms ptm overhead 34 mpu 68
Sent 1010872782 bytes 4230140 pkt (dropped 2113, overlimits 588813 requeues 0)
backlog 0b 0p requeues 0
memory used: 371072b of 4Mb
capacity estimate: 17Mbit
min/max network layer size: 28 / 1492
min/max overhead-adjusted size: 70 / 1550
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 1062Kbit 17Mbit 8500Kbit 4250Kbit
target 17.1ms 5ms 5ms 5ms
interval 112ms 100ms 100ms 100ms
pk_delay 406us 156us 156us 335us
av_delay 47us 30us 24us 110us
sp_delay 19us 27us 20us 27us
backlog 0b 0b 0b 0b
pkts 784594 3415407 95 32157
bytes 54717959 954873313 6553 4320695
way_inds 2203 77350 0 0
way_miss 2361 49290 40 491
way_cols 0 0 0 0
drops 0 2113 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 1 2 1 1
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 2928 67872 394 1278
quantum 300 518 300 300
qdisc clsact ffff: dev pppoe-wan parent ffff:fff1
Sent 31617342449 bytes 41269637 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8024: dev ifb-pppoe-wan root refcnt 2 bandwidth 56Mbit diffserv4 dual-dsthost nonat nowash ingress no-ack-filter split-gso rtt 100ms ptm overhead 34 mpu 68
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 0b of 4Mb
capacity estimate: 56Mbit
min/max network layer size: 65535 / 0
min/max overhead-adjusted size: 65535 / 0
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 3500Kbit 56Mbit 28Mbit 14Mbit
target 5.19ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 0us 0us 0us 0us
av_delay 0us 0us 0us 0us
sp_delay 0us 0us 0us 0us
backlog 0b 0b 0b 0b
pkts 0 0 0 0
bytes 0 0 0 0
way_inds 0 0 0 0
way_miss 0 0 0 0
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 0 0
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 0 0 0 0
quantum 300 1514 854 427
ip addr
12: dsl0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP qlen 1000
link/ether dc:39:6f brd ff:ff:ff:ff:ff:ff
46: ifb-dns: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UNKNOWN qlen 32
link/ether 12:4b:75:df:18:b0 brd ff:ff:ff:ff:ff:ff
inet6 fe80::104b:75ff:fedf:18b0/64 scope link
valid_lft forever preferred_lft forever
50: dsl0.101@dsl0: <BROADCAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP qlen 1000
link/ether dc:39:6f brd ff:ff:ff:ff:ff:ff
51: pppoe-wan: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1492 qdisc cake state UNKNOWN qlen 3
link/ppp
inet 86.147.XX.XXX PU|BLIC IP peer 172.16.11.165/32 GW scope global pppoe-wan
valid_lft forever preferred_lft forever
inet6 fe80::3c33:c156:be49:b8c1/128 scope link
valid_lft forever preferred_lft forever
55: ifb-pppoe-wan: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc cake state UNKNOWN qlen 32
link/ether c6:ec:6e brd ff:ff:ff:ff:ff:ff
inet6 fe80::c4ec:6eff:fe1d:7d88/64 scope link
valid_lft forever preferred_lft forever
Do you have qosify installed? If so, stop and disable the service and then reboot the router⌠had similar issues with dscpclassify (which works similar as cake-qos-simple) and qosifyâŚ
I think qosify doesnât delete all tc related settings properlyâŚ
That would make sense then, Yes i have used qosify in the past but i disabled qosify a long time ago.
Will start with a fresh install and got back to us.
Also check the filters like I mentioned earlier.
tc filter show dev pppoe-wan parent ffff:
If qosify is the issue, wouldn't just removing qosify be enough?
Iâd just like to see where the download traffic is being mirred to if not the ifb interface. Iâm speculating the filter failed because kmod-sched-ctinfo isnât installed, or something like that.
This is from qosify⌠should probably be ingress
Sorry, i wiped the router to fresh, Im stuck on Snapshots at the moment as the DSL modem in this router isnt on 22.03.X release.
Ok, i think i have a few errors.
root@R1:~# /etc/init.d/cake-qos-simple start
Setting up ingress handle for dev pppoe-wan.
No dl_if specified, so setting up appropriate IFB.
Creating IFB device: ifb-pppoe-wan.
Setting interface ifb-pppoe-wan up.
Setting up tc filter to restore DSCPs from conntrack on ingress packets on interface pppoe-wan and redirecting to IFB interface ifb-pppoe-wan.
Setting up CAKE on interface pppoe-wan with bandwidth 17Mbit/s and options: diffserv4 dual-srchost nonat wash no-ack-filter pppoe-ptm ether-vlan noatm.
Setting up CAKE on interface ifb-pppoe-wan with bandwidth 54Mbit/s and options: diffserv4 dual-dsthost nonat nowash ingress no-ack-filter pppoe-ptm ether-vlan noatm.
Started cake-qos-simple.
root@R1:~# service firewall restart
In file included from /dev/stdin:20:2-33:
/etc/nftables.d/cake-qos-simple.nft:9:1-5: Error: syntax error, unexpected table
table inet cake-qos-simple
^^^^^
In file included from /dev/stdin:20:2-33:
/etc/nftables.d/cake-qos-simple.nft:10:1-6: Error: syntax error, unexpected delete
delete table inet cake-qos-simple
^^^^^^
In file included from /dev/stdin:20:2-33:
/etc/nftables.d/cake-qos-simple.nft:24:1-5: Error: syntax error, unexpected table
table inet cake-qos-simple {
^^^^^
/dev/stdin:27:14-14: Error: syntax error, unexpected '{', expecting string or last
chain input {
^
/dev/stdin:28:3-6: Error: syntax error, unexpected type
type filter hook input priority filter; policy drop;
^^^^
/dev/stdin:28:43-48: Error: syntax error, unexpected policy
type filter hook input priority filter; policy drop;
^^^^^^
/dev/stdin:30:3-9: Error: syntax error, unexpected iifname
iifname "lo" accept comment "!fw4: Accept traffic from loopback"
^^^^^^^
/dev/stdin:32:6-10: Error: syntax error, unexpected state, expecting timeout or expectation or helper
ct state established,related accept comment "!fw4: Allow inbound established and related flows"
^^^^^
/dev/stdin:33:6-10: Error: syntax error, unexpected state, expecting timeout or expectation or helper
ct state invalid drop comment "!fw4: Drop flows with invalid conntrack state"
^^^^^
/dev/stdin:34:3-9: Error: syntax error, unexpected iifname
iifname "br-lan" jump input_lan comment "!fw4: Handle lan IPv4/IPv6 input traffic"
^^^^^^^
The rendered ruleset contains errors, not doing firewall restart.
The nft file is supposed to be in /usr/share/nftables.d/ruleset-post/
Okay. This wasnt the issue i had before i restarted with a fresh copy.
root@R1:~# /etc/init.d/cake-qos-simple download
qdisc cake 8006: root refcnt 2 bandwidth 54Mbit diffserv4 dual-dsthost nonat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 34
Sent 116510737 bytes 86621 pkt (dropped 3389, overlimits 158374 requeues 0)
backlog 0b 0p requeues 0
memory used: 370816b of 4Mb
capacity estimate: 54Mbit
min/max network layer size: 36 / 1492
min/max overhead-adjusted size: 70 / 1526
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 3375Kbit 54Mbit 27Mbit 13500Kbit
target 5.38ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 8.1ms 42us 0us 165us
av_delay 3.49ms 22us 0us 5us
sp_delay 35us 9us 0us 5us
backlog 0b 0b 0b 0b
pkts 1831 88165 0 14
bytes 1980964 119583114 0 2833
way_inds 0 1 0 0
way_miss 22 338 0 4
way_cols 0 0 0 0
drops 2 3387 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 6 0 1
bk_flows 0 1 0 0
un_flows 0 0 0 0
max_len 1385 1492 0 1385
quantum 300 1514 823 411
root@R1:~# /etc/init.d/cake-qos-simple upload
qdisc cake 8005: root refcnt 2 bandwidth 17Mbit diffserv4 dual-srchost nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 34
Sent 24605103 bytes 74564 pkt (dropped 1034, overlimits 30754 requeues 0)
backlog 14920b 10p requeues 0
memory used: 97408b of 4Mb
capacity estimate: 17Mbit
min/max network layer size: 29 / 1492
min/max overhead-adjusted size: 63 / 1526
average network hdr offset: 0
Bulk Best Effort Video Voice
thresh 1062Kbit 17Mbit 8500Kbit 4250Kbit
target 17.1ms 5ms 5ms 5ms
interval 112ms 100ms 100ms 100ms
pk_delay 546us 13.5ms 50us 237us
av_delay 64us 6.23ms 0us 9us
sp_delay 15us 877us 0us 9us
backlog 0b 14920b 0b 0b
pkts 718 74855 3 32
bytes 155644 26002104 87 4916
way_inds 0 1 0 0
way_miss 22 344 1 6
way_cols 0 0 0 0
drops 0 1034 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 2 1 1
bk_flows 0 4 0 0
un_flows 0 0 0 0
max_len 1385 16886 29 462
quantum 300 518 300 300
qdisc ingress ffff: parent ffff:fff1 ----------------
Sent 123508395 bytes 100445 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
So is it working?
Crikey the link at the top has over a hundred hits now. That's cool!
Only issue i have now, is the speed feel slow.
Compare this with Qosify i get much better speed if i set port 8080 to CS0 as what speedtest uses.
I don' think that makes sense, but of course you could just set port 8080 packets to cs0 if you wanted to. @dave14305 or @moeller0 any ideas?
without it
Not related to the speed, but why use nonat if youâre using dual-dsthost and dual-srchost? CAKE wonât lookup the real internal host IP.
what is your best suggestion for a
VDSL2, FTTC with VLAN101 been used over PPPoE @moeller0 might best to ask? since your on cable/docsis, right @dave14305
@Lynx
This what i did with Qosify to max port 8080
/etc/qosify
config defaults
list defaults /etc/qosify/*.conf
option dscp_default_tcp bulk
option dscp_default_udp bulk
config class besteffort
option ingress CS0
option egress CS0
config class bulk
option ingress LE
option egress LE
# VDSL2 with a ISP using VLAN 101 over FTTC
option ingress_options "ptm overhead 34 mpu 68"
option egress_options "ptm overhead 34 mpu 68"
option options "ether-vlan"
00-defaults.conf
tcp:8080 besteffort
udp:8080 besteffort
Then used http://ipv4.download.thinkbroadband.com:8080/1GB.zip to max out the ports leaving bulk with abot 3Mb since best effort was prioritized more.
Excellent point - it should be nat in both directions right? I'll make that commit now.
@francisuk1989 DSCPs will only help when you are trying to push through more bandwidth than your connection can support, and cake has to decide what to sacrifice. So you can tell cake not to sacrifice Teams or Zoom or XBOX games. Does that make sense?
cs0 is just best effort, isn't it?
Unless youâre putting cake on the LAN interface after traffic has been de-natted. But with the current default behavior of the script, I think itâs the sane default.
1 Like