Hello all!
My VDSL connection is pretty slow, with 27.000Kbps download and 6.700Kbps upload, but it has enough upload to do live-streaming at 1080p with 4500Kbps bitrate. The only issue is that I need to prioritize that traffic, otherwise if another devices uploads something, I start to loose frames.
To do that I configured this traffic rule on Firewall settings:
And then I configured SQM on WAN interface like this:
But it does not seem to work. I've checked with Wireshark and packets that gets sent have "Destination Port 1935", with protocol TCP.
For testing I just tried to run a Speedtest on my phone: when it starts the upload test, stream speed decreases from 4500Kbps to about 800Kbps, so something is clearly wrong.
Any advice?
mpa
July 1, 2020, 12:09pm
2
Check which cake tin the traffic was assigned to:tc -s qdisc ls dev eth0.835
Thanks for your answer!
This is the output I get... while streaming the "bytes" row of the "Best Effort" column increase a lot. Not sure how to interpret it.
qdisc cake 8011: root refcnt 2 bandwidth 6500Kbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 34
Sent 21749665 bytes 16738 pkt (dropped 84, overlimits 27138 requeues 0)
backlog 0b 0p requeues 0
memory used: 125312b of 4Mb
capacity estimate: 6500Kbit
min/max network layer size: 28 / 1492
min/max overhead-adjusted size: 62 / 1526
average network hdr offset: 14
Bulk Best Effort Voice
thresh 406248bit 6500Kbit 1625Kbit
target 44.8ms 5.0ms 11.2ms
interval 139.8ms 100.0ms 106.2ms
pk_delay 0us 19.6ms 736us
av_delay 0us 8.4ms 24us
sp_delay 0us 795us 24us
backlog 0b 0b 0b
pkts 0 16806 16
bytes 0 21866564 2114
way_inds 0 1 0
way_miss 0 131 6
way_cols 0 0 0
drops 0 84 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 1 1
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 31626 269
quantum 300 300 300
mpa
July 1, 2020, 12:19pm
4
Now do the same check for the speed test.
Yes, exactly. I don't think that RTMP traffic should be in the "Best Effort" tin, right?
mpa
July 1, 2020, 12:52pm
6
Right, but the higher-priority Voice tin appears to be limited to 1625Kbit.
I've done a little bit more researching, and found out that Tins are decided by the "diffserv" setting. Default seems to be diffserv3, I set it to diffserv8 by editing the /usr/lib/sqm/defaults.shfile and recreating the sqm interface.
Now I can see 8 tins... will test and let you know! Thanks for pointing this out to me!
Not without editing the cake kernel code and recompilation. But you can much easier change simple.qos, there the three implemented priority tiers are completely constructed/defined in /usr/lib/sqm/simple.qos.
Just make a copy of simple.qos and simple.qos.help to say my_simple.qos and my_simple.qos.help and edit both. They then will appear in the list of queue scripts....
1 Like
You still can not manually set the min and max bandwidth for the tins in cake...
And there is your issue, SQM/cake default to per flow fairness (with potential per-internal-IP-fairness on top) and 4.5 Mbps simply is more than the fair share of your bandwidth. These easiest way around that is to created a modified copy of /usr/lib/sqm/simple.qos. Have a look at:https://forum.openwrt.org/t/sqm-cake-and-being-a-twitch-streamer-upload-direction/62474/13?u=moeller0
Thanks for your answer!
I copied simple.qos to a new file (twitch.qos) and edited the PRIO_RATE value on the egress() function to PRIO_RATE=5000.
I have then selected the new qos from the list (keeping "cake" in the queue discipline), but it's not working at all.
Running tc -s qdisc ls dev eth0.835 returns:
qdisc cake 8052: root refcnt 2 bandwidth 6300Kbit diffserv3 triple-isolate nonat nowash no-ack-filter split-gso rtt 100.0ms noatm overhead 34
Sent 109434075 bytes 91331 pkt (dropped 846, overlimits 129150 requeues 0)
backlog 1506b 1p requeues 0
memory used: 197760b of 4Mb
capacity estimate: 6300Kbit
min/max network layer size: 28 / 1492
min/max overhead-adjusted size: 62 / 1526
average network hdr offset: 14
Bulk Best Effort Voice
thresh 393744bit 6300Kbit 1575Kbit
target 46.3ms 5.0ms 11.6ms
interval 141.3ms 100.0ms 106.6ms
pk_delay 0us 27.3ms 1.9ms
av_delay 0us 10.2ms 204us
sp_delay 0us 342us 46us
backlog 0b 1506b 0b
pkts 0 92081 99
bytes 0 110665793 10794
way_inds 0 1 0
way_miss 0 270 20
way_cols 0 0 0
drops 0 846 0
marks 0 0 0
ack_drop 0 0 0
sp_flows 0 0 2
bk_flows 0 1 0
un_flows 0 0 0
max_len 0 31626 1392
quantum 300 300 300
qdisc ingress ffff: parent ffff:fff1 ----------------
Sent 23365994 bytes 67326 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
and stream traffic does not get prioritized. I get better results by setting differv8 (still some drops as not all packets gets top priority but just some of them).
Well, sorry for not mentioning that, you need to use fq_codel here, this is not set up for cake as leaf qdisc...
Still no luck... my speedtest takes over 4.5Mbps causing the stream to drop a lot of frames.
I did change the DSCP classification to 46 as per the tutorial (Note: I've also tried EF):
And set the options like this on SQM:
This is the edited part of the egress() function on twitch.qos:
Thanks
What is the output of:tc -s qdisctc -d qdisccat /etc/config/sqm
Also, I believe that you will need to adjust the divisors for the other classes up, so that the sum of PRIO_RATE, BE_RATE and BK_RATE stays below ceiling...
Time for packet captures
root@Main-Router:/usr/lib/sqm# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 3047824377 bytes 3430210 pkt (dropped 0, overlimits 0 requeues 360)
backlog 0b 0p requeues 360
maxpacket 1514 drop_overlimit 0 new_flow_count 221 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc htb 1: dev br-app root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 direct_qlen 1000
Sent 12781 bytes 124 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 120: dev br-app parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 1636 bytes 37 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 124 drop_overlimit 0 new_flow_count 2 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc fq_codel 130: dev br-app parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev br-app parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 11145 bytes 87 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 135 drop_overlimit 0 new_flow_count 67 ecn_mark 0
new_flows_len 1 old_flows_len 0
qdisc ingress ffff: dev br-app parent ffff:fff1 ----------------
Sent 10669 bytes 169 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.2 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth0.1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev eth0.835 root refcnt 2 r2q 10 default 0x12 direct_packets_stat 68 direct_qlen 1000
Sent 833876889 bytes 675446 pkt (dropped 2925, overlimits 1020588 requeues 0)
backlog 4532b 2p requeues 0
qdisc fq_codel 120: dev eth0.835 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 833433333 bytes 674168 pkt (dropped 2925, overlimits 0 requeues 0)
backlog 4532b 2p requeues 0
maxpacket 34936 drop_overlimit 0 new_flow_count 26703 ecn_mark 0 memory_used 7392
new_flows_len 0 old_flows_len 1
qdisc fq_codel 130: dev eth0.835 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 110: dev eth0.835 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 156220 bytes 1050 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 1426 drop_overlimit 0 new_flow_count 723 ecn_mark 0
new_flows_len 1 old_flows_len 7
qdisc ingress ffff: dev eth0.835 parent ffff:fff1 ----------------
Sent 95147180 bytes 461160 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 105505 bytes 1254 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev tap0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 2635747 bytes 16536 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 0 drop_overlimit 0 new_flow_count 0 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev wlan1-1 root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc htb 1: dev ifb4eth0.835 root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 direct_qlen 32
Sent 117242820 bytes 461134 pkt (dropped 26, overlimits 74680 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4eth0.835 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 117242820 bytes 461134 pkt (dropped 26, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 1540 drop_overlimit 0 new_flow_count 32290 ecn_mark 0
new_flows_len 0 old_flows_len 1
qdisc htb 1: dev ifb4br-app root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 direct_qlen 32
Sent 13035 bytes 169 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 110: dev ifb4br-app parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
Sent 13035 bytes 169 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
maxpacket 130 drop_overlimit 0 new_flow_count 160 ecn_mark 0
new_flows_len 1 old_flows_len 0
root@Main-Router:/usr/lib/sqm# tc -d qdisc
qdisc noqueue 0: dev lo root refcnt 2
qdisc fq_codel 0: dev eth0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc htb 1: dev br-app root refcnt 2 r2q 10 default 0x12 direct_packets_stat 0 ver 3.17 direct_qlen 1000
qdisc fq_codel 120: dev br-app parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 130: dev br-app parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 110: dev br-app parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc ingress ffff: dev br-app parent ffff:fff1 ----------------
qdisc noqueue 0: dev eth0.2 root refcnt 2
qdisc noqueue 0: dev br-lan root refcnt 2
qdisc noqueue 0: dev eth0.1 root refcnt 2
qdisc htb 1: dev eth0.835 root refcnt 2 r2q 10 default 0x12 direct_packets_stat 68 ver 3.17 direct_qlen 1000
linklayer ethernet overhead 34
qdisc fq_codel 120: dev eth0.835 parent 1:12 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 130: dev eth0.835 parent 1:13 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 110: dev eth0.835 parent 1:11 limit 1001p flows 1024 quantum 300 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc ingress ffff: dev eth0.835 parent ffff:fff1 ----------------
qdisc noqueue 0: dev wlan1 root refcnt 2
qdisc noqueue 0: dev wlan0 root refcnt 2
qdisc fq_codel 0: dev tun0 root refcnt 2 limit 10240p flows 1024 quantum 1500 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc fq_codel 0: dev tap0 root refcnt 2 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc noqueue 0: dev wlan1-1 root refcnt 2
qdisc htb 1: dev ifb4eth0.835 root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 ver 3.17 direct_qlen 32
linklayer ethernet overhead 34
qdisc fq_codel 110: dev ifb4eth0.835 parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
qdisc htb 1: dev ifb4br-app root refcnt 2 r2q 10 default 0x10 direct_packets_stat 0 ver 3.17 direct_qlen 32
qdisc fq_codel 110: dev ifb4br-app parent 1:10 limit 1001p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
root@Main-Router:/usr/lib/sqm# cat /etc/config/sqm
config queue 'eth1'
option qdisc 'fq_codel'
option script 'simple.qos'
option qdisc_advanced '0'
option linklayer 'none'
option interface 'br-app'
option download '3400'
option upload '8200'
option debug_logging '0'
option verbosity '5'
option enabled '1'
config queue
option debug_logging '0'
option verbosity '5'
option enabled '1'
option interface 'eth0.835'
option download '26000'
option qdisc_advanced '0'
option linklayer 'ethernet'
option overhead '34'
option upload '6000'
option qdisc 'fq_codel'
option script 'twitch.qos'
I've installed tcpdump-mini on the router, and trying to do what you asked.
I found this command online, but no packet gets captured.tcpdump -nni br-lan -v 'ip[1] & 0xfc == 46'
By listening normally with tcpdump -n -i br-lan
I get these packets:
16:50:12.190771 IP 192.168.12.110.60999 > 52.223.200.13.1935: Flags [.], seq 493344:494796, ack 17, win 64108, length 1452
16:50:12.190951 IP 192.168.12.110.60999 > 52.223.200.13.1935: Flags [P.], seq 494796:496600, ack 17, win 64108, length 1804
Not sure how to check if the DSCP flag is set.
Ok, found a way to export to file, analyzed it with Wireshark and... nope. DSCP Value is AF43 for those packets, and not EF
I tried everything... seems that all my settings to get DSCP tagging are ignored
Well, are the marked packets directed to the marked port? Maybe you could try to set the dscps on your twitch streaming machine first.
It seems so
I did tried to set up dscp on my machine, but seems it's not working either... I've used this command on a superuser powershell:
New-NetQosPolicy -Name "twitch" -AppPathNameMatchCondition "obs64.exe" -PolicyStore ActiveStore -NetworkProfile All -DSCPAction 46
try to generate some traffic that should be marked 46 or EF and take a packet capture on your router while the streaming runs. Make sue just capture for a short while or attach some additional storage to the router, as most routers are not build for storing megabytes of data. Then look at the capture file with, say, wireshark and conform that the packets are correctly dscp marked....
