Cake QOS R7800, expectations

Hi all,

After having considerable issues with my ISP provided router I decided to go for the R7800 with a OpenWRT install, I have installed the latest hnyman build and then installed Cake using the piece of cake option. This took me to A+ A+ A on DSLreports, normally what would happen on my network is that someone would be on a Xbox One and then it would update and patch a game and everyone would suffer and buffer until this had finished. Expecially when watching videosm it wouldn't be possible.

So I set a test going with the Xbox downloading and then trying to stream some videos, it worked and the video no longer buffered. I noticed that the download for the Xbox kept starting and stopping until it went into a failed queue on there to start after the next update had downloaded.

I thought nothing of it and carried on, I moved onto another test by starting a file download from the PC and see if video can still be streamed as this is another issue in the house. I set a HTTPS download going in NZBGET and tried to see if two streams would play whilst this was happening, one would play, the other would just stop and not continue until the file download was stopped.

I have 64Mbps download and 20Mbps upload, the file download was getting around 62Mbps, it was going up and down but not by a massive amount, pretty soon after one video would start the second one would stop.The devices were wireless but both had >700Mbps connecion rate to the router.

As I am new to this I had just hoped that this would share the connection fairly between all the devices but it could be that I just don't understand what the hell it should be and is doing which is far more likely.

Any guidance would be appreciated, thanks...

Something sounds strange as it should be possible to get multiple streams going without stalling out, unless your downloads are starving, but no streaming I've heard of goes at 64Mbps it's much more typical less than 10Mbps

Can you post a screenshot of your SQM setting in Luci? that'd be a good place to start.

Definitely not the behaviour that I observe on my R7800. Is it possible that SQM is instanciated on the wrong interface?

Hi, Thanks for the replies

I have currently got the AC86U plugged back in as my networking device so the X4S hasn't got an internet connection. I can only upload one picture at the moment as a new user so I will include just the log.

Also what I meant with the 64Mbps, I had a download running on a Newsgroup to test and then I was trying to get two video streams to playback whilst the download was happening.

I have the QOS running on the WAN interface as far as im aware, eth0.2 (wan)

IMG-0112

QOS%201

Can you post a link to the results page from a dslreports speedtest with those settings?

Sure;

So, that looks normal.

The behavior you mention about streams or downloads stalling, can you reproduce it consistently? or is it spotty?

I haven't used Usenet since the 1900's :slight_smile: but I looked up NZBGET and it says:

"Unlike downloads from www or ftp the download from usenet requires quite a lot of computation work. NZBGet is optimized for performance and uses very little memory at the same time."

Perhaps you have a CPU issue on your computer streaming two HD videos and, doing a Usenet download could be an issue for computing power?

I can reproduce it yes, I was downloading on the PC and trying to watch two twitch streams. One on an iPad pro and the other on a Galaxy S8, one of the devices would just stop and the others would continue. As soon as I stopped the download for the PC the other stream would continue playing back. The Usenet download didn't seem to be getting throttled that much to be honest.

I would say an average twitch stream is up to about 10Mbps at most but the gaming machine was still running at about 44Mbps. This is what made me think that I was expecting either too much or I didn't know how it should be working.

The traffic is HHTPS encrypted but I don't think that should matter should it, its all about fair sharing with Cake?

Yes, but perhaps you want per-internal-ip sharing, since my guess is your Usenet downloader probably opens up many many different connections, and so per-stream fairness is going to be giving most of the bandwidth to the downloader. per-ip fairness would require a few minor tweaks, @moeller0 is the expert there but you can start by looking at wiki: https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details#making_cake_sing_and_dance_on_a_tight_rope_without_a_safety_net

1 Like

The other option would be to somehow identify those streams and tag them with DSCP=CS1 and use a more complicated configuration described in excruciating detail: Ultimate SQM settings: Layer_cake + DSCP marks

1 Like

I will take a look at the links thankyou, the Newsgroup downloader is set to 8 as the maximum number of connections to the server. Would this be enough to make a difference?

Yes, as by default all streams get similar bandwidth, so your 8 download streams + 2 video streams would get 80% of the bandwidth to downloading and 10% to stream on iPad, 10% stream on phone. Whereas per-ip-fairness would get 33% to downloading 33% to stream on iPad, 33% to stream on Phone.

1 Like

Thank you that makes much more sense to me now, I will have a look at setting up the per-ip-fairness as that does indeed sound like what I want, going forward also looking at the second post that mentions tagging the streams as ideally I would want to substantially reduce any downloading traffic.

Looking at the wiki I am liking the look of " Per-Host Isolation" as that seems to deal with the issue that I am having with multiple connections taking more than their fair share.

I will give that a try now!

Also the last point that I must mention as it is strange to me as that I seem to be getting better range and throughput with the AC86U that I do with the X4S where all the reviews that I have read state that the X4S has better range and throughput which I find strange.

Per host isolation will solve your problem as stated, but you'll still have the issue within the one computer that does the usenet download. So if you try to stream a video there, you'd maybe get 8/9 of the bandwidth going to the download and 1/9 going to your video stream.

In that case you can maybe cut the number of simultaneous connections to say 2 or 3 and avoid this problem, or you can try to tag those streams and use diffserv4 or something. You may also have bandwidth limiting settings on your downloader, you could request that it uses no more than say 20Mbps at any given time, or during certain hours or whatever, will take longer to download, but since you're probably doing long downloads anyway, it's not something you're necessarily waiting on so its all good.

I entered the code as stated in the wiki but it made no difference and the buffering on both video streams was almost instant as soon as I started the download.

Here are the statistics of the traffic control after running the download and the two streams;

root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc mq 0: dev eth0 root
 Sent 118132479 bytes 579578 pkt (dropped 0, overlimits 0 requeues 13082)
 backlog 0b 0p requeues 13082
qdisc fq_codel 0: dev eth0 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 118132479 bytes 579578 pkt (dropped 0, overlimits 0 requeues 13082)
 backlog 0b 0p requeues 13082
  maxpacket 1514 drop_overlimit 0 new_flow_count 8536 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc mq 0: dev eth1 root
 Sent 1013670174 bytes 926835 pkt (dropped 0, overlimits 0 requeues 19369)
 backlog 0b 0p requeues 19369
qdisc fq_codel 0: dev eth1 parent :1 limit 10240p flows 1024 quantum 1514 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 1013670174 bytes 926835 pkt (dropped 0, overlimits 0 requeues 19369)
 backlog 0b 0p requeues 19369
  maxpacket 1514 drop_overlimit 0 new_flow_count 14253 ecn_mark 0
  new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev eth1.1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8004: dev eth0.2 root refcnt 2 bandwidth 17Mbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 118131653 bytes 579571 pkt (dropped 1812, overlimits 105275 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 1Mb of 4Mb
 capacity estimate: 17Mbit
 min/max network layer size:           24 /    1514
 min/max overhead-adjusted size:       24 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         17Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay        111us
  av_delay         13us
  sp_delay          8us
  backlog            0b
  pkts           581383
  bytes       120862900
  way_inds         2593
  way_miss         4563
  way_cols            0
  drops            1812
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum           518

qdisc ingress ffff: dev eth0.2 parent ffff:fff1 ----------------
 Sent 1457720839 bytes 1053214 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc cake 8005: dev ifb4eth0.2 root refcnt 2 bandwidth 60Mbit besteffort dual-dsthost nat wash no-ack-filter split-gso rtt 100.0ms raw overhead 0
 Sent 1464595474 bytes 1048008 pkt (dropped 5206, overlimits 1170090 requeues 0)
 backlog 0b 0p requeues 0
 memory used: 420Kb of 4Mb
 capacity estimate: 60Mbit
 min/max network layer size:           60 /    1514
 min/max overhead-adjusted size:       60 /    1514
 average network hdr offset:           14

                  Tin 0
  thresh         60Mbit
  target          5.0ms
  interval      100.0ms
  pk_delay        1.6ms
  av_delay        362us
  sp_delay         10us
  backlog            0b
  pkts          1053214
  bytes      1472465835
  way_inds        28551
  way_miss         4768
  way_cols            0
  drops            5206
  marks               0
  ack_drop            0
  sp_flows            0
  bk_flows            1
  un_flows            0
  max_len          1514
  quantum          1514

qdisc fq_codel 0: dev pppoe-wan root refcnt 2 limit 10240p flows 1024 quantum 1518 target 5.0ms interval 100.0ms memory_limit 4Mb ecn
 Sent 108070640 bytes 581234 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
  maxpacket 1492 drop_overlimit 0 new_flow_count 51 ecn_mark 0
  new_flows_len 1 old_flows_len 0
qdisc noqueue 0: dev wlan1 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
qdisc noqueue 0: dev wlan0 root refcnt 2
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
 backlog 0b 0p requeues 0
root@OpenWrt:~#

These are the settings I am using on my R7800 with a PPPoE connection 50/10: no video buffering during any downloads. Your overhead value might be different, though. What is your connection to your ISP?

config queue 'wan'
	option debug_logging '0'
	option verbosity '5'
	option linklayer 'ethernet'
	option linklayer_advanced '1'
	option tcMPU '64'
	option qdisc_advanced '1'
	option ingress_ecn 'ECN'
	option egress_ecn 'NOECN'
	option qdisc_really_really_advanced '1'
	option iqdisc_opts 'nat dual-dsthost mpu 64'
	option eqdisc_opts 'nat dual-srchost mpu 64'
	option enabled '1'
	option interface 'pppoe-wan'
	option qdisc 'cake'
	option script 'piece_of_cake.qos'
	option squash_dscp '0'
	option squash_ingress '0'
	option linklayer_adaptation_mechanism 'default'
	option tcMTU '2047'
	option tcTSIZE '128'
	option overhead '34'
	option download '47500'
	option upload '9500'
1 Like

I have 64Mbps down currently and 20Mbps up, here is my config;

config queue 'eth0'
        option linklayer 'none'
        option enabled '1'
        option download '60000'
        option upload '17000'
        option interface 'eth0.2'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option debug_logging '1'
        option qdisc_advanced '1'
        option squash_dscp '1'
        option squash_ingress '1'
        option ingress_ecn 'ECN'
        option egress_ecn 'NOECN'
        option qdisc_really_really_advanced '1'
        option iqdisc_opts 'nat dual-dsthost'
        option eqdisc_opts 'nat dual-srchost'
        option verbosity '8'

root@OpenWrt:~#