Cake QOS R7800, expectations

How does this look, this is my connection when my link is being fully saturated.
pingplotter

Well, supposing this is the behavior for say RTP packets for a VOIP connection, you'd have basically completely unbroken audio with 40ms jitter buffer (2 audio packets). Since most VOIP implementations probably use 60 to 80 ms jitter buffer to start with (and adapt it upward if needed), I think it should be good enough for anything but the most demanding competitive gamer :wink:

1 Like

To get that sort of setup I would imagine it would be setting up priorities for gaming traffic, like you stated above. I think as a standard setup this is as good as I am going to get, Iā€™m not complaining either.

1 Like

Other than marking traffic is there anything else to try to lower latency?

Is there an easier explaination for this;

Cake actually offers the "ingress" keyword to better deal with shaping ingress links (it will for example not scale its packet dropping so the shaper's egress rate matches the set rate, but rather so that the ingress rate matches the set rate, which works better for post-bottle-neck shaping). IIRC, "ingress" also will better deal with the number of concurrent flows on the ingress link (more flows typically require a higher bandwidth sacrifice so that all flows get a sufficiently strong signal to slow down), and it dies this transiently depending on the number of active flows. In short, if you use the ingress keyword you might get away with a smaller permanent bandwidth sacrifice, but that still is policy and you need to balance this for your own network according to your preferences.

Iā€™m trying but I donā€™t understand, other than it will be better.

I can try to rephrase things and elaborate, maybe you should ask explicit questions you want answered after reading the above?

Anyway in short:
Download shaping is a bit approximate, as the shaper is on the wrong end of the true bottle-neck link. Why does this matter? Well, the problem that makes download shaping appealing in the first place typically is the fact that the buffers in the machine sitting at the other (upstream) end of the bottleneck link (the dslam in your case, let's just call it the head-end) are over-sized and/or under-managed. The trick now is to avoid filling those "bloated" buffers in the first place and do all the buffering in the right-sized and properly managed buffers on our (downstream) end of the bottle-neck.
The challenge with this is, that if too much data rushes in to head-end in too shot a time, those packets will pile up in the head-end buffers and will cause noticeable latency-under-load increases (aka bufferbloat).
Typically this issue gets less and less likely and noticeable the larger the difference in bandwidth exist between the true bottleneck rate and the shaper rate (and many users seem satisfied with the experienced bufferbloat-effects when shaping down to 85-90% of the true link rate) but this is only approximate as the efficiency depends on the behavior and number of incoming data-flows. Why you ask? Well, traditional egress shapers drop enough packets so that the rate of packets leaving the shaper matches the shaper rate, but the incoming rate will be higher. The issue with post-bottleneck-shaping is that in is the rate of incoming packets that should stay below the bottleneck capacity (on average) to avoid undesirable "back-spill" into the head-end's buffers.
The cake shaper can be instructed with the keyword ingress to aim at shaping the incoming rate to the configured value and hence deals better with the issues laid out above.

I hope this clarifies things a bit...

2 Likes

Marking traffic is the best way, because not all traffic needs to be low latency, so knowing what does and what doesn't allows you to delay and drop non sensitive traffic so that sensitive traffic can go first... Think about an ambulance on the street, others pull over and stop so they can go through at high speed.

1 Like

Thank you both for the replies, that does clear things up for me.

Does the ingress keyword get its value from the bandwidth setting?

Yes, according to @moeller0 it just changes what's being monitored from the egress rate towards the LAN instead it watches the ingress rate from the WAN.

Do you have some particular issues with latency that are problematic? Your plot showed pretty good latency under load, good enough that VOIP should work well and that's one of the most sensitive types of flows. If you run an office with multiple phones or you play action games you might want to start working on marking traffic.

Prior to implementing Cake on my home network we would run into all sorts of issues with devices hogging the bandwidth, either game consoles starting downloads or photos uploading to the cloud when connected to the power at night. This meant that video streaming or gaming was always getting interrupted. I have no particular issue with latency at the moment I just wanted to make sure it was as optimised as it could be.

Here are the current settings;

qdisc cake 8007: dev pppoe0 root refcnt 2 bandwidth 17Mbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100.0ms ptm overhead 30

qdisc cake 8010: dev ifb4pppoe0 root refcnt 2 bandwidth 59Mbit besteffort dual-dsthost nat nowash ingress no-ack-filter split-gso rtt 100.0ms ptm overhead 30

Hopefully this looks right, initially I had an issue with Usenet so enabled Per-Host fairness which alleviated that issue and I moved the Usenet onto a NAS drive which further removed that issue. As a gamer I was just looking to make sure I was getting the best latency I could get, even when lowering bandwidth now I see diminishing returns on the latency. I will do more testing though.

Looks decent, personally I dislike the ptm keyword though and just make sure the configured rate stays at <= 100*64/65 = 98.46% of the sync rate. That avoids one division cake needs to perform and s actually more precise than what cake does internally (cake errs on the conservative side, so wastes a ever so tiny bit of bandwidth). Also I like diffserv3 better than besteffort (but this will only have an effect if the packets seen by cake have "proper" diffserv markings...)

I am currently using pppoe-ptm due to my VDSL 2 connection, is this not correct?

If I use diffserv3 do I have to do all the markings? Or could some traffic be marked by default? Is there nothing to lose in trying it, if the traffic isnā€™t marked does it just act like besteffort?

I am currently running at 55Mbps on my download bandwidth and my latency hasnā€™t spiked above 20ms since.

If cake diffserv3 only ever sees packets with the default dscps of 0 (or unmarked packets) it will behave exactly as besteffort, albeit with a slightly higher computational cost IIRC.

That is not wrong, except the point about that I prefer to statically reduce the shaper rate once instead of doing a division/modulo operation per packet, but that is just a personal preference. The bigger question is, does your ISP actually use a VLAN tag on your link or not? If yes, then I believe the overhead should be 34 instead of 30.

I found this which would suggest that it uses a VLAN,

How do I input the 34 rather than using the pppoe-ptm command?

Also it would seem like using diffserv3 canā€™t hurt anything other than maybe a bit more cpu, I donā€™t really understand how to go about marking traffic myself but how would traffic be marked by default? Is that down to the ISP?

The program creating the traffic controls the mark initially. Then at any hop along the way it could change. So on ingress it would be rare for DSCP to be meaningful. Your LAN is under your own control so it makes the most sense to mark traffic to your liking at ingress. Unfortunately the iptables rules for ingress don't get run until after the ingress queue is already processed. (nftables solves this problem but isn't compatible with the default firewall on OpenWrt).

On egress to WAN on the other hand, you have some control of all the hops from the device that produces the packet through to your router. Some games are marking packets already, but you can also remark them during routing.

If you're a gamer, and you're still not happy with your QoS, marking packets is your best bet, but it requires a bit of a deep dive... certainly deeper than filling in some boxes on LuCI. Here's the current mega-thread on developing marked packet strategies: Ultimate SQM settings: Layer_cake + DSCP marks

2 Likes

Could you repost the output of
cat /etc/config/sqm
again please, then I can post the changed version here.

In the SQM-GUI just head over to the "Link Layer Adaptation" tab, select
Ethernet (with overhead) as "Which link layer to account for:"
put 34 into "Per Packet Overhead (byte):"
set " Minimal packet size, MPU (byte); needs to be > 0 for ethernet size tables:" to 68
" Which linklayer adaptation mechanism to use; for testing only" to "default"

then make sure to delete the pppoe-ptm keyword from "Advanced option string to pass to the ingress queueing disciplines; no error checking, use very carefully." and "Advanced option string to pass to the egress queueing disciplines; no error checking, use very carefully."

I note that most recent sqm-scripts actually will give some ERROR feedback when calls to tc fail that might be useful to debug the advanced option string keywords, for sufficiently motivated users :wink:

Does this look ok with the updated Overhead, I still have tpm on for the VDSL;

qdisc cake 8015: dev pppoe0 root refcnt 2 bandwidth 17Mbit besteffort dual-srchost nat nowash no-ack-filter split-gso rtt 100.0ms ptm overhead 34

qdisc cake 8017: dev ifb4pppoe0 root refcnt 2 bandwidth 55Mbit besteffort dual-dsthost nat nowash no-ack-filter split-gso rtt 100.0ms ptm overhead 34

I cant currently get onto the GUI so I did the updated settings through putty.

Yes it does.

Your choice, effectively this results in the true shaper gross-rates being <= 17*64/65 = 16.74 Mbps and 55*64/65 = 54.15 Mbps. Personally I prefer to simply factor this reduction directly into the shaper gross rates as that allows simpler predictions about the maximum achievable goodput, but as I said it is a matter of taste.

As an aside,

for ADSL (actually ATM/AAL5 encoding is the relevant part here, in theory one could use ptm on an ADSL link (and ptm was introduced in an Annex for one of the ADSL flavors IIRC)) such a static prospective rate reduction is not possible. Now, you might ask, if for PT'M's 64/65-encoding all we need to do is to reduce the sync rate by 100-10064/65 = 1.54% why can't we do the same for ATM's 48/53 encoding (100-10048/53 = 9.43%)? The reason is simply that the PTM encoding happens independent of whether there is any user data, any 65th byte will be used by PTM, while ATM's encoding happens on a packet by packet basis, as each packet needs to to be encapsulated into an integer number of 48-byte ATM cells. This means that the effective length of an ATM packet depends on its actual length. To give an example, a data packet of 48 bytes fits into a single ATM cell, but a packet of 49 bytes will required two cells with the second one mostly being occupied by padding bytes (the will give you an approximate ATM overhead of 100-10049/(532) = 53.77% clearly the predicted 9.4% from above will be a bad fit)

1 Like

Thank you @moeller0 I appreciate it and I will remove ptm and enter the bandwidth figures that you state later, thank you again.

I donā€™t think I have asked this here but, when setting bandwidth to 55Mbps I get about 51Mbps on download. Is the 4Mbps cake overhead?