Cake QOS R7800, expectations


#21

Mmh, I assume this is a VDSL (aka Fibre to the Curb/Cabinet) link in the UK, correct?
In that case, I wonder whether you connect the router to a modem that does PPPoE?
(I guess the overhead values need a bit of tuning, but that most likely is not causing your observed issue.)
If that is true could you please get a reading of the modem's link parameters (the synchonization values with the dslam are off interest here)?
Finally you could try adding the keyword ingress to iqdisc_opts:
option iqdisc_opts 'nat dual-dsthost ingress'
That should make the bandwidth sacrifice automatically more-or-less scale with the number of concurrent incoming flows, which might slightly ameliorate your problem.


#22

HI all,

I managed to get this sorted by restarting the config from scratch, after I did that and re-entered the commands I now have it working as intended. by default now if anything tried to take too much bandwidth, like the PC for example it is restricted and the other devices get a fair share of the bandwidth.

Should I expect the wireless coverage to be less when using OpenWRT? I have both the AC86U and the X4S at the moment as I am deciding which one to keep. With the routers in the exact same place when using 2.4ghz my Ring doorbell goes from a signal of -53db to -62db. Also my 5ghz is weaker than it is on the AC86U which seems to contradict the reviews for the X4S being the best in that regard.


#23

Use speed tests to determine coverage rather than simply signal strength. Stand where your Ring device is and run a speed test on your phone or a tablet. Is there a difference?

Beamforming and mu-mimo and etc can be more relevant, and they are dynamic technologies that vary with conditions.


#24

OT, but some info about the Ring devices: https://www.macrumors.com/2019/01/10/ring-employees-customer-camera-access/


#25

Thanks for the Ring info, looks shady on their part to say the least.

I still have both devices so I will have to put them to the test against each other, not sure what the best software to use is to measure. I have an iPad pro so I could just do a DSLreports speedtest from various locations maybe. I think that the X4S is holding its own for the most part, but in certain areas the AC86U is just better for some reason.

I have been playing around with the QOS tonight and it is working nicely now, I can have 5-6 streams going with a Usenet download going without issue. If I try and play a video on the PC with the Usenet download then the streams on that computer are the only ones to buffer as it is limiting the bandwidth the PC can have as a device compared to the others.

I like Cake QOS as it seems very fluid in operation and on its own without user intervention it does a good job, whereas the AC86U has to have a QOS script to make the Trend MIcro solution work well and also certain stuff needs re-tagging, however once that is done it does allow for more things to happen on the same bandwidth hungry devices but only if the traffic is tagged properly. I see that I can could use something like FireQOS but then that looks complicated compared to the FreshJR QOS script for the AC86U.


#26

Cake can respond to tags also, it is a bit more technical to get this to work if the tags aren't already there and need to be placed there by you.


#27

If you fall back to standard qos-scripts (luci-app-qos), which uses hsfc+fq_codel, you can do prioritization by TCP port. This may or may not work better in your situation. You would give up the per host fairness of cake, you might win by lowering the priority of your usenet . If you try this, you have to uninstall SQM.

SQM simple.qos may allow this too - I can't remember and can't check right now.

Alternatively, you could go with a separate download box (e.g. raspberry pi or similar). That would be my approach.


#28

Well, what cake tries to do is isolate the different computers in your network from each other, so this is somewhat expected... For most users this seems to be an acceptable trade-off. IMHO layer_cake in combination with marking your Usenet downloads CS1 (background) should get you better interactivity, but you need to figure out how to do the marking, which can be tricky for ingress. Well or maybe the Usenet download can be run from a container/VM with its own IP-address as then cake will isolate it for you. Finally, I would not be amazed if the Usenet download application could be configured with a maximum rate to reign into it's worst-case behavior....


#29

Hi, the Usenet downloader can be throttled, it was more just a case of testing out as I’m trying to decide which router to keep between the X4S and the AC86. I already know how to mark traffic on the QOS the Asus uses, however that isn’t exactly a set and forget option as although it does a good job, occasionally things get tagged into the wrong category and then the QOS makes it worse. For the most part the FreshJR QOS script fixes any issues, also fq_codel has been added into the AC86U but I don’t think that it is fully supported by the software.

I think as an overall solution Cake is great, the Asus can allow more with tagging, as can the X4S if I learn how to do it properly. I need to do more testing between the two routers to see which one has the lowest latency overall when taxed, I’m assuming cake will win out.

I have moved Usenet to a Nas device with all the Usenet software running via Docker so that should isolate it from doing too much damage and mean it doesn’t need to be throttled as downloads are mostly dynamic.

What’s the best way to test wireless speed between routers in various locations, I have an iPad Pro that I can use with iPerf3 maybe?


#30

If you can set up an iperf3 server then hurricane electrics network tools program can be the client. I know there's an Android version, maybe an iPad version too, don't know.


#31

Thank you for the information, Network tools looks great I have looked before for something similar but not found anything.

This is from my iPad in the same room as the router on 5ghz. image


#32

Also any idea what would have caused this to happen, I thought that my internet went down as devices lost connection but I'm not sure what happened now, Wifi radio issue?

Fri Jan 11 20:20:20 2019 daemon.notice netifd: Network device 'wlan1' link is down
Fri Jan 11 20:20:20 2019 kern.info kernel: [123823.852387] device wlan1 left promiscuous mode
Fri Jan 11 20:20:20 2019 kern.info kernel: [123823.852623] br-lan: port 3(wlan1) entered disabled state
Fri Jan 11 20:20:20 2019 kern.warn kernel: [123823.894728] ath10k_pci 0001:01:00.0: peer-unmap-event: unknown peer id 1
Fri Jan 11 20:20:20 2019 kern.warn kernel: [123823.894818] ath10k_pci 0001:01:00.0: peer-unmap-event: unknown peer id 1
Fri Jan 11 20:20:20 2019 kern.warn kernel: [123823.900561] ath10k_pci 0001:01:00.0: peer-unmap-event: unknown peer id 1
Fri Jan 11 20:20:21 2019 user.notice mac80211: Failed command: iw phy phy1 set antenna all all
Fri Jan 11 20:20:21 2019 daemon.notice netifd: radio1 (14758): command failed: Not supported (-95)
Fri Jan 11 20:20:21 2019 user.notice mac80211: Failed command: iw phy phy1 set distance 0
Fri Jan 11 20:20:21 2019 daemon.err hostapd: Configuration file: /var/run/hostapd-phy1.conf
Fri Jan 11 20:20:27 2019 kern.info kernel: [123830.565624] IPv6: ADDRCONF(NETDEV_UP): wlan1: link is not ready
Fri Jan 11 20:20:27 2019 kern.info kernel: [123830.587058] br-lan: port 3(wlan1) entered blocking state
Fri Jan 11 20:20:27 2019 kern.info kernel: [123830.587117] br-lan: port 3(wlan1) entered disabled state
Fri Jan 11 20:20:27 2019 kern.info kernel: [123830.592146] device wlan1 entered promiscuous mode
Fri Jan 11 20:20:27 2019 daemon.err hostapd: Using interface wlan1 with hwaddr 8c:3b:ad:b6:c5:8a and ssid "Nighthawk 2.4"
Fri Jan 11 20:20:28 2019 kern.info kernel: [123831.016292] IPv6: ADDRCONF(NETDEV_CHANGE): wlan1: link becomes ready
Fri Jan 11 20:20:28 2019 kern.info kernel: [123831.016667] br-lan: port 3(wlan1) entered blocking state
Fri Jan 11 20:20:28 2019 kern.info kernel: [123831.021745] br-lan: port 3(wlan1) entered forwarding state
Fri Jan 11 20:20:28 2019 daemon.notice netifd: Network device 'wlan1' link is up

#33

Can someone help me with the following as I can’t work it out as a relative newcomer to all this.

Why wouldn’t you want Per-host isolation on by default, why wouldn’t that be the default option in cake?

What does triple isolate do by default that makes it the default?

Is cake just assuming that for most people, each person will be doing one internet thing on said device and with that in mind triple-isolate will do for the most part the right thing?

However in a more complicated environment where more things are happening at once, mostly PC users that maybe doing multiple internet based things at once, like downloading, streaming or gaming for example on the same device that they want to enable Per-host to avoid it affecting the other users?

How would triple isolate handle the following;

1 x video stream iPad

1 x video stream phone

1 x video stream kids tab

1 x Pc download 5 connections + gaming traffic on same machine.

I feel like there is something missing from my understanding of how things actually works.


#34

Well, you need information about the direction of the interface cake runs on in relation to the internet, otherwise you might end up requesting per-external-IP-address fairness which rarely is what users desire...

Triple isolate does away with the strict directionality requirement of the dual-host modes by simultaneously aiming for both src- and host-address fairness, which for normal use-cases is a good approximation for per-internal-IP-address fairness (except in ectreme cases like speedtests where it behaves differently).

In essence yes, that is yes to the "triple-isolate will do for the most part the right thing" part... I believe the exact retionale is slightly different, but you get the point...

Nah, my rationale for recommending the dual-isolation modes is that it should be easier to predict how they behave in most situations, while triple-isolate's behavior is less tangible.

Honestly, I beieve it should just work okay, except for the "1 x Pc download 5 connections + gaming traffic on same machine." host. Per-IP-address fairness will most likely not help in running games and downloads on the same machine concurrently. IMHO that is one of these "Doctor it hurts when I do X; -> don't do X then" kind of situations. Either that or a deep dive into DSCP marking territory :wink:

Just run a few experiments and see whether triple-isolate works well enough for your use-case then? In reality these are all policy decisions about trade-offs that need to be tested against each individual networks requirements...


#35

In my experience, SQM (cake or fq_codel) work perfectly fine when doing more than one thing on a single device. The per flow fairness takes care of this. The issue is when you have something that both opens numerous flows AND each of those uses a lot of bandwidth. So torrents or other multi-stream downloads. In the grand scheme, this is semi-common corner case, but one that would ideally be solved eventually.

By far the best current solution is to put these multi-stream bandwidth hogs on their own device - Raspberry Pi or a NAS are great for this. RPi is cheap. The per IP fairness will ensure these downloads don't suck up all the bandwidth.

Or you could just throttle the downloads in the client - either with a cap or reducing the maximum connections.


#36

Thank you for the replies,

From my testing I would have to use Per-host fairness as stated above as greedy users/devices can disrupt others users without this setting on my network.

1 x twitch stream iPhone
1 x twitch stream iPad
1 x twitch stream and 8 x Usenet download on PC

Without Per-host fairness the twitch stream that buffers is random between all the clients, with Per-host only the gaming PC twitch stream is affected by the download.

Even if I moved the Usenet to a NAS drive I’m not sure how Xbox, PlayStation or Switch downloads are handled. If they are multiple streams or just one, so I think for me Per-host is the only way. There could always be one device that could possibly spawn multiple streams with high bandwidth and effect the whole network, rather than the one greedy device.


#37

I handle these cases with DSCP marking and downgrading long transfers, it takes a bit more custom setup though, if isolation works it is v easy


#38

I will look into that as a project going forward I think, that’s sort of what I was doing with the AC86U and the custom QOS script that they have and the fq_codel sort of support as I think the AsusWRT kernel version is way behind Openwrt. I love the simplicity of Cake, with Per-host it’s almost a perfect solution for me. Only the distruptive client gets throttled, very very nice.

The only thing I didn’t like was if something got tagged by default into the wrong category and then that makes the whole connection worse. However when you do it yourself rather than implementing custom commands onto Trend Micros solution that is probably not going to happen.

Do you know of a good resource for learning how to go about implementing DSCP markings?


#39

That is a setup similar to mine where each device (IP) is specialized. Each of them gets a fair share and I can tell that heavy cloud uploads/downloads from one IP, do not affect PS4 gaming experience or surfing the Internet from any wireless/wired device. My connection is slow at 50/10 so a R7800 with SQM can happily keep up with the load.

Update: I am not using any DSCP markings.


#40

I seem to be having issues with the 5ghz wireless on the R7800, not sure why but it doesn’t seem very stable. I had a look in the kernel log and I’m seeing this;

[300883.153798] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.648315] ath10k_warn: 6 callbacks suppressed
[344241.648329] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.652033] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.660218] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.668452] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.676669] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode
[344241.684977] ath10k_pci 0001:01:00.0: received unexpected tx_fetch_ind event: in push mode

I don’t lose WiFi connection but the router becomes unresponsive as if the internet has gone down which it hasn’t. 2.4ghz seems unaffected.

image