qosify 2021-11-12-bfc2cafe
Please try to update the qosify package. A new snapshot build with the latest version just finished.
Is there any guide I can follow to tailor this software to my needs? Thanks in advance.
so on wrt3200acm
/etc/init.d/qosify status shows running
however no process or tc shows qdisc
running qosify on its own
libbpf: map 'config': created successfully, fd=4
libbpf: failed to mkdir /sys/fs/bpf/qosify_data/config: No such file or directory
libbpf: map 'config': failed to auto-pin at '/sys/fs/bpf/qosify_data/config': -2
libbpf: map 'config': failed to create: No such file or directory(-2)
libbpf: failed to load object '/lib/bpf/qosify-bpf.o'
bpf_object__load: No such file or directory
Thanks! Now I really need to get my old test router in service again so I can build and test this.
Why? It should be enough to simply not enable sqm on the same interface. One of the things sqm-scripts has going for it is that it allows multiple instances on different interfaces (which occasionally is actually useful).
Thanks for the hint. That approach makes it really easy to include qosify in a build.
Documenting the steps for others:
-
download the llvm-bpf-13.0.0.tar.xz from your router's target downloads to your buildroot. E.g. https://downloads.openwrt.org/snapshots/targets/ipq806x/generic/llvm-bpf-13.0.0.tar.xz
-
extract the files into the buildroot:
tar -xvf llvm-bpf-13.0.0.tar.xz -
Change toolchain option to use a prebuilt LLVM toolchain, and enable qosify. I only needed to add these two lines into my
.configrecipe, and all other dependencies got pulled in automatically:CONFIG_USE_LLVM_PREBUILT=y CONFIG_PACKAGE_qosify=y
If you have extracted the LLVM toolchain properly, it will get detected by make menuconfig or defconfig, and the related options get set correctly in .config. If the toolchain is missing, the modified config settings will not show up in .config.
4 Likes
There's nothing in qosify that conflicts with sqm-scripts. I mentioned this only because many people get confused thinking that qosify is an add-on to sqm-scripts and that they need both.
3 Likes
Which OpenWrt version are you using? Does /sys/fs/bpf exist on your device, and what's in it?
master + /sys/fs/bpf exists but nothing under
Ah, sure makes sense then!
Is anything mounted to /sys/fs/bpf? Normally it should be mounted by default. Are you using extroot or anything like that?
yes extroot ...
Has anybody else observed (after ~2 days of uptime) bufferbloat is getting worse? After restart of qosify it came back from B grade to A.
Freshly after restart of the router
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 2170802 bytes 8212 pkt (dropped 0, overlimits 0 requeues 5)
backlog 0b 0p requeues 5
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 2170802 bytes 8212 pkt (dropped 0, overlimits 0 requeues 5)
backlog 0b 0p requeues 5
maxpacket 1494 drop_overlimit 0 new_flow_count 9 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
Sent 7133960 bytes 13935 pkt (dropped 0, overlimits 0 requeues 12)
backlog 0b 0p requeues 12
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 7133960 bytes 13935 pkt (dropped 0, overlimits 0 requeues 12)
backlog 0b 0p requeues 12
maxpacket 1514 drop_overlimit 0 new_flow_count 107 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-guest root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.3 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev eth0.2 root refcnt 2 bandwidth 20Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 2165107 bytes 8177 pkt (dropped 2, overlimits 2004 requeues 0)
backlog 0b 0p requeues 0
memory used: 26688b of 4Mb
capacity estimate: 20Mbit
min/max network layer size: 40 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 1250Kbit 20Mbit 10Mbit 5Mbit
target 14.6ms 5ms 5ms 5ms
interval 110ms 100ms 100ms 100ms
pk_delay 552us 1.8ms 2.11ms 49us
av_delay 13us 129us 227us 22us
sp_delay 9us 26us 18us 12us
backlog 0b 0b 0b 0b
pkts 24 55 6465 1635
bytes 9897 56322 1799252 301196
way_inds 0 0 66 0
way_miss 6 5 502 276
way_cols 0 0 0 0
drops 0 0 2 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 1 1 2 2
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 1817 5682 16434 1514
quantum 300 610 305 300
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
Sent 732456 bytes 2677 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev ifb-eth0.2 root refcnt 2 bandwidth 300Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 772556 bytes 2677 pkt (dropped 0, overlimits 54 requeues 0)
backlog 0b 0p requeues 0
memory used: 45596b of 15000000b
capacity estimate: 300Mbit
min/max network layer size: 46 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 18750Kbit 300Mbit 150Mbit 75Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 16us 68us 53us 46us
av_delay 0us 2us 17us 18us
sp_delay 0us 2us 11us 15us
backlog 0b 0b 0b 0b
pkts 7 31 1777 862
bytes 432 12325 639784 120015
way_inds 0 0 0 0
way_miss 5 4 362 268
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 2 3
bk_flows 0 1 1 0
un_flows 0 0 0 0
max_len 66 4684 31794 590
quantum 572 1514 1514 1514
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
After a few hours
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 937392645 bytes 2365930 pkt (dropped 0, overlimits 0 requeues 8)
backlog 0b 0p requeues 8
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 937392645 bytes 2365930 pkt (dropped 0, overlimits 0 requeues 8)
backlog 0b 0p requeues 8
maxpacket 1514 drop_overlimit 0 new_flow_count 12 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
Sent 7638111748 bytes 6628810 pkt (dropped 7, overlimits 0 requeues 392)
backlog 0b 0p requeues 392
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 7638111748 bytes 6628810 pkt (dropped 7, overlimits 0 requeues 392)
backlog 0b 0p requeues 392
maxpacket 1514 drop_overlimit 0 new_flow_count 216324 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-guest root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.3 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev eth0.2 root refcnt 2 bandwidth 20Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 937386950 bytes 2365895 pkt (dropped 4844, overlimits 851310 requeues 0)
backlog 0b 0p requeues 0
memory used: 1244032b of 4Mb
capacity estimate: 20Mbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 1250Kbit 20Mbit 10Mbit 5Mbit
target 14.6ms 5ms 5ms 5ms
interval 110ms 100ms 100ms 100ms
pk_delay 1.02ms 1.22ms 1.59ms 246us
av_delay 119us 188us 195us 119us
sp_delay 11us 44us 10us 12us
backlog 0b 0b 0b 0b
pkts 2206 4397 2053990 310146
bytes 561317 3134528 902615759 38062268
way_inds 0 0 76427 39071
way_miss 123 375 41113 16714
way_cols 0 0 0 0
drops 0 5 4839 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 1 2 2
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 43479 17205 46934 1594
quantum 300 610 305 300
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
Sent 366249771 bytes 551041 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev ifb-eth0.2 root refcnt 2 bandwidth 300Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 374557159 bytes 550973 pkt (dropped 68, overlimits 33727 requeues 0)
backlog 0b 0p requeues 0
memory used: 1591208b of 15000000b
capacity estimate: 300Mbit
min/max network layer size: 46 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 18750Kbit 300Mbit 150Mbit 75Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 31us 77us 1.67ms 40us
av_delay 7us 22us 1.19ms 18us
sp_delay 7us 15us 10us 12us
backlog 0b 0b 0b 0b
pkts 144 511 397572 152814
bytes 17256 312977 357148477 17176441
way_inds 0 0 4929 13151
way_miss 69 89 24856 12034
way_cols 0 0 0 0
drops 0 0 68 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 0 3 1
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 1229 12415 68130 1514
quantum 572 1514 1514 1514
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
Restart of qosify service helped only partially. I was able from time to time to get A bufferbloat mark. Restart of the router however resulted with stable bufferbloat A grade.
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
Sent 1037462454 bytes 2492300 pkt (dropped 0, overlimits 0 requeues 8)
backlog 0b 0p requeues 8
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 1037462454 bytes 2492300 pkt (dropped 0, overlimits 0 requeues 8)
backlog 0b 0p requeues 8
maxpacket 1514 drop_overlimit 0 new_flow_count 14 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
Sent 7682827426 bytes 6672486 pkt (dropped 7, overlimits 0 requeues 392)
backlog 0b 0p requeues 392
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 7682827426 bytes 6672486 pkt (dropped 7, overlimits 0 requeues 392)
backlog 0b 0p requeues 392
maxpacket 1514 drop_overlimit 0 new_flow_count 216364 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-guest root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.3 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev eth0.2 root refcnt 2 bandwidth 20Mbit diffserv4 dual-srchost nat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 201455 bytes 992 pkt (dropped 0, overlimits 145 requeues 0)
backlog 5936b 4p requeues 0
memory used: 15104b of 4Mb
capacity estimate: 20Mbit
min/max network layer size: 40 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Bulk Best Effort Video Voice
thresh 1250Kbit 20Mbit 10Mbit 5Mbit
target 14.6ms 5ms 5ms 5ms
interval 110ms 100ms 100ms 100ms
pk_delay 0us 915us 1.14ms 73us
av_delay 0us 23us 111us 7us
sp_delay 0us 13us 23us 7us
backlog 0b 0b 5936b 0b
pkts 0 80 813 103
bytes 0 13090 181370 12931
way_inds 0 0 0 0
way_miss 0 5 85 25
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 1 2 1
bk_flows 0 0 1 0
un_flows 0 0 0 0
max_len 0 5682 3146 1505
quantum 300 610 305 300
qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
Sent 29856 bytes 183 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 1: dev ifb-eth0.2 root refcnt 2 bandwidth 300Mbit diffserv4 dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 32418 bytes 183 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
memory used: 2Kb of 15000000b
capacity estimate: 300Mbit
min/max network layer size: 46 / 1470
min/max overhead-adjusted size: 64 / 1488
average network hdr offset: 7
Bulk Best Effort Video Voice
thresh 18750Kbit 300Mbit 150Mbit 75Mbit
target 5ms 5ms 5ms 5ms
interval 100ms 100ms 100ms 100ms
pk_delay 0us 18us 29us 24us
av_delay 0us 4us 4us 1us
sp_delay 0us 4us 4us 1us
backlog 0b 0b 0b 0b
pkts 0 76 86 21
bytes 0 8924 20720 2774
way_inds 0 0 0 0
way_miss 0 5 30 12
way_cols 0 0 0 0
drops 0 0 0 0
marks 0 0 0 0
ack_drop 0 0 0 0
sp_flows 0 4 2 0
bk_flows 0 0 0 0
un_flows 0 0 0 0
max_len 0 590 1484 373
quantum 572 1514 1514 1514
i restart every 24h ... master always has memory leak I find ...
Could could please perform the following test:
-
start qosify fresh.
-
immediately run the following command on your routers command line and store the output somewhere:
tc -s qdisc -
let things run until bufferbloat gets worse then do not (yet) restart qosify but run:
tc -s qdisc
again -
restart qosify and run
tc -s qdisc
again -
post the output of the three call to tc here in the thread, ideally by following these instructions below:
Please use the "Preformatted text </>" button for logs, scripts, configs and general console output.
Please edit your post accordingly. Thank you! 
The goal is to figure out whether the cake instances my disappear.... (for example if you use pppoe-wan your ISP might initiate PPPoE disconnects after which OpenWrt might tear pppoe-wan down and set it fresh up again, after which cake needs to be re-instantiated). I might be barking up the wrong tree here, but I would be happy if you could run that test even though it likely will not help.
In point 3 you can still use another class to configure the bulk priority?
hi everybody I have configured qosify like this
/etc/config/qosify
Summary
`config defaults
list defaults /etc/qosify/*.conf
option dscp_prio CS5
option dscp_icmp +CS0
option dscp_bulk LE
option dscp_default_udp CS4
option bulk_trigger_timeout 5
option bulk_trigger_pps 100
option prio_max_avg_pkt_len 500
config class bulk
option ingress LE
option egress LE
config class video
option ingress AF41
option egress AF41
config class voice
option ingress CS6
option egress CS6
option bulk_trigger_pps 100
option bulk_trigger_timeout 5
option dscp_bulk CS0
config interface wan
option name wan
option disabled 0
option bandwidth_up 16mbit
option bandwidth_down 56mbit
# defaults:
option ingress 1
option egress 1
option mode diffserv4
option nat 1
option host_isolate 1
option autorate_ingress 1
option ingress_options ""
option egress_options ""
option options "overhead 44"
config device wandev
option disabled 1
option name wan
option bandwidth 100mbit`
I added a script so that it starts automatically when starting the router in
/etc/hotplug.d/iface add file qosify and put this in
Summary
`#!/bin/sh
[ -n "$DEVICE" ] || exit 0
[ "$ACTION" = ifup ] && {
echo y | /etc/init.d/qosify reload | start
logger -t qosify "Reloading qosify due to $ACTION of $INTERFACE ($DEVICE)"
}`
and finally i has add my rules in iptables
like this
Summary
`iptables -t mangle -A POSTROUTING -p udp --dst 192.168.2.135 -j DSCP --sport 30000:45000 --dport 3074 --set-dscp-class CS5 -m comment --comment "Dopam-IT_1987-UDP-1-CALL-OF-DUTY"
iptables -t mangle -A POSTROUTING -p udp --src 192.168.2.135 -j DSCP --sport 3074 --dport 30000:45000 --set-dscp-class CS5 -m comment --comment "Dopam-IT_1987-UDP-2-CALL-OF-DUTY"
iptables -t mangle -A POSTROUTING -p tcp --dst 192.168.2.135 -j DSCP --dport 50000:65535 --sport 3074 --set-dscp-class CS5 -m comment --comment "Dopam-IT_1987-TCP-3-CALL-OF-DUTY"
iptables -t mangle -A POSTROUTING -p tcp --src 192.168.2.135 -j DSCP --dport 3074 --sport 50000:65535 --set-dscp-class CS5 -m comment --comment "Dopam-IT_1987-TCP-4-CALL-OF-DUTY"
iptables -t mangle -A POSTROUTING -p udp --src 192.168.2.135 -j DSCP --sport 3074 --dport 3074 --set-dscp-class CS5 -m comment --comment "cod-UDP-6-CALL-OF-DUTY"
##FIFA APEX LEGENDS EA GAMES BATTLEFIELD
iptables -t mangle -A POSTROUTING -p udp --dst 192.168.2.135 -j DSCP --sport 10000:30000 --dport 3659 --set-dscp-class CS5 -m comment --comment "FIFA-UDP-1-BATTLEFIELD2042"
iptables -t mangle -A POSTROUTING -p udp --src 192.168.2.135 -j DSCP --sport 3659 --dport 10000:30000 --set-dscp-class CS5 -m comment --comment "FIFA-UDP-2-BATTLEFIELD2042B"`
my dslreports is amazing and for first impression for the moment my game and network is very good ..
1 Like
the other thing happening is the following message
procd: Instance qosify::instance1 s in a crash loop 6 crashes, 0 seconds since last crash
i tried to install the new snapshot and the new qosify package but still the same error
Command failed: Request timed out
any other suggestions?
libbpf: failed to load program 'classify'
libbpf: unpinned map 'config' from '/sys/fs/bpf/qosify_data/config'
libbpf: unpinned map 'flow_map' from '/sys/fs/bpf/qosify_data/flow_map'
libbpf: unpinned map 'tcp_ports' from '/sys/fs/bpf/qosify_data/tcp_ports'
libbpf: unpinned map 'udp_ports' from '/sys/fs/bpf/qosify_data/udp_ports'
libbpf: unpinned map 'ipv4_map' from '/sys/fs/bpf/qosify_data/ipv4_map'
libbpf: unpinned map 'ipv6_map' from '/sys/fs/bpf/qosify_data/ipv6_map'
libbpf: unpinned map 'class_map' from '/sys/fs/bpf/qosify_data/class_map'
libbpf: failed to load object '/lib/bpf/qosify-bpf.o'
bpf_object__load: Invalid argument