SQM high latency

Currently sqm-scripts does not pass tcMPU to cake, iqdisc_opts 'mpu 64'/eqdisc_opts 'mpu 64' will sidestep this. Once sqm-script and luci-app-sqm is fixed the difference will basically be that iqdisc_opts/eqdisc_opts have no error checking and are hence not well suited to casual users. At the current state you will need to use iqdisc_opts 'mpu 64' for layer_cake/piece_of_cake and option tcMPU '64' for simplest/simple.

Hope this helps...

Yes, it does but in this case why would you recommend him to put this in his config?

'option iqdisc_opts 'nat dual-dsthost'
'option eqdisc_opts 'nat dual-srchost'

Would be enough.

Oh, so as you explained I know i've understood wrong. I though i have a pppoe connection so i set overhead to 8. Is 34 suitable value for a pppoe connection (my type is fiber)?

I really want to say Thank You to ALL members of LEDE and all associated members that have contributed to such a wonderful distro. Also to the the bufferbloat contributors for their big efforts as well. Without them we wouldn't have the great range of software to use today. I do think some solid and working platform for donations should be posted. Even as little as $5-$10 a person that downloads could go a long way. I'm not saying to enforce donations, but these men and women work hard to create the building blocks for better networking systems. I honestly can't remember the last time I've witnessed so much effort being put into making something work efficiently as this in 20 years of my IT experience.

After tuned with overhead value to 26 (like you) i see no difference. Do i need to keep new value (or tune again) ? I really confuse with overhead things :confused:

The point is, we do not know. There are two interrelated questions about overhead we need to differentiate. On first sight the answer is simple, figure out the real "hardware-level" overhead that each packet incurs on the bottleneck link; a decent assumption is that this overhead should depend on the used link technology as well as additionally used encapsulations. On second look this only is relevant if the bandwidth available to the user is actually limited by link technology (as it often is with old ADSL lines), many more modern technologies like cable (DOCSIS) and fiber (GPON/EPON/...) actualy offer way more bandwidth than the usual contract allows the user to use. In that common case the relevant overhead for sqm-scripts purposes is not (necessarily) the on-link overhead, but rather the overhead that the traffic shaper/policer the ISP uses to restrict users to the contracted bandwidth is configured for. Case in point the actual overhead on DOCSIS packets is quite large, but the DOCSIS mandated shaper only accounts for 18 bytes and hence for sqm-scripts those 18 bytes are the correct overhead to configure.
Unfortunately it is not that straight forward to actually measure the relevant per-packet overhead especially on non-ATM links and not all ISPs give their users information about the current gross bandwidth making measuring the overhead quite hard. (If you configure the overhead in sqm-scripts higher than it really is, you will still have good bufferbloat reduction, you will just sacrifice a bit more bandwidth than you would otherwise require, conversely if you set the bandwidth sufficiently lower than the true bottleneck bandwidth you wil still see great bufferbloat reduction; in other words peak useable bandwidth and overhead are not really independent measures). If you trust that you know any one of them well, you can use that knowledge to determine the other (in both cases if set the known parameter to its correct value then if you systematically vary the other at one point you will see a sharp increase in bufferbloat (well, actually latency under load will increase sharply)). Doing so it a hassle though, so for non-ATM links I am not sure I would recommend going that route.*
Now, I am sorry that this is not the answer to your question, but it is the best answer I can give you...

*) Since per Paket Overhead is fixed the way to disentangle unknown Max bandwidth and unknown overhead is to use packets of increasing sizes, but I digress....

So in my case empirically the assumption of 1526 byte frames worked out. But this is a VDSL2 link AND I learned that mybISPs traffic shaper also assumes 1526 byte frames... In my case I do not know exactly what the ISPs shaper is set to, but at least the overhead I trust :slight_smile:

Best Regards

Moeller0

Does Fq_Codel and/or Cake automtically subtract 5% for both directions by default?

No, at least not on purpose...

To elaborate (from a different response with less visibility, re-posted here, because maybe it is of interest to others as well):

Quick note, the shaper bandwidth is gross bandwidth, so you will need to take some overheads into account to get to the achievable good-put. Most speedtests actually seem to report good-put so there is a discrepancy between shaper setting and speedtest results.

So, for your 18 bytes overhead on MTU 1500 byte TCP/IPv4 packets you will see at least a 100-100*((1500-20-20)/1518) = 3.8208168643% lower goodput than gross shaper rate. If you use a lower MTU (not fully under your control, with path MTU discovery and partial link between you and the speedtest servers can result in a lower MTU) or more overheads (say TCP timestamps https://www.ietf.org/rfc/rfc1323.txt) you will see even more difference. See http://www.speedguide.net/analyzer.php for information about currently used TCP options, this will also report an MTU value, but that while interesting will not tell you the pMTU to the speedtest servers, so do not take this too seriously, also I would not recommend to take the rest of the speedguide recommendations too serious; it is a nice way to confirm TCP timestamps and other options.
In addition the these 3.8% percent apparent "loss" you most likely will also see some reduction from the fact that a single TCP connection is typically not able to constantly saturate a link (due to TCPs back-off behavior when encountering loss); this "loss" will get smaller the more concurrent measuring flows you use...
(I would recommend the dslreports speedtest, which after a free registration allows individual configurations of the speedtest, my recommendations: enable High-Res bufferbloat testing and compression dodging, set the number of ingress/egress streams to 8 or 16 (start with 16 and scale back to 8 if you get errors during the test), and finally set the ingress/egress test duration to >= 30 seconds). The nice thing about the dslreports test is that you will get a detailed result page where you can see the latency measurements of the individual latency probes (and this forum allows users to post links to the detailed results pages making sharing/discussing things quite convenient)

Hope that helps...

I am noticing this too on an Archer C7 2.0

I believe that it has something to do with the SQM process itself.

My current profile is 250/20 DOCSIS 3.0. In SQM I set the download speed to 0 as suggested in the forums. The reason I took this advice is because I saw that when using top -d 1 at the speed of 250 the sirq # would hit 99 due to no hardware NAT present.

The interesting thing is, if SQM is on even with download set at 0, the bufferbloat # in ms is noticably higher than when SQM is turned off entirely.

I don't have insight into the inner workings, but I think that the packets are still going through a SQM workflow even if the download is set to 0. Ideally, when set to 0, the packets should flow through the normal route as if SQM was not activated at all so they don't have to be processed or touched at all. I.e. SQM script for egress only.

Is there a way to configure it to be egress only but not ingress?

Below are my current /etc/config/sqm settings:

config queue 'eth1'
    option debug_logging '0'
    option verbosity '5'
    option qdisc 'cake'
    option script 'piece_of_cake.qos'
    option upload '19000'
    option enabled '1'
    option download '0'
    option linklayer 'ethernet'
    option qdisc_advanced '1'
    option squash_dscp '1'
    option squash_ingress '1'
    option egress_ecn 'NOECN'
    option qdisc_really_really_advanced '1'
    option linklayer_advanced '1'
    option tcMTU '2047'
    option tcTSIZE '128'
    option tcMPU '64'
    option overhead '18'
    option ingress_ecn 'ECN'
    option interface 'eth0'
    option linklayer_adaptation_mechanism 'default'
    option iqdisc_opts 'mpu 64 nat dual-dsthost'
    option eqdisc_opts 'mpu 64 nat dual-srchost'

This is way too much bandwidth for your poor C7 to handle...

This is exactly the right thing to do, have a look at /usr/lib/sqm/piece_of_cake.qos to convince yourself that nothing nefarious is happening when download = 0, als compare the output of "tc -s qdisc" with option download '0' and option download '50000'. No real idea why you still see worse performance with download '0' though (so I am not doubting your observation, I am just pointing out that this is not something by design)

I have 120/10Mb connection on my WR1043NDv4 (same CPU as in Archer C7). When download is set to 0 my pings/latency is spiking up and the latency in general is not that great when maximizing the download.

I don't think there is anything wrong with your setup. It's just because SQM can't do the shaping on download you will notice some latency issues.

Pings/latency spiking is caused by the cablemodem itself if it's a Puma6 chipset.

Huge thread here on the subject: https://www.dslreports.com/forum/r31122204-SB6190-Puma6-TCP-UDP-Network-Latency-Issue-Discussion
First post on this other thread has graphs demonstrating it: https://www.dslreports.com/forum/r31079834-ALL-SB6190-is-a-terrible-modem-Intel-Puma-6-MaxLinear-mistake

Because of this (my cablemodem is a Puma6 chipset) I cannot talk about pings/latency in this thread.

To be fair, I can only observe the sirq # inside the router at different speeds and bufferbloat.

It turns out I also have Puma 6 chipset based modem from UPC.
It's a shame... I don't think they will fix this any time soon.

Thanks for the tip regarding the Puma 6 chipset. It turns out it affects Thomson DCM475 modems as well.

Replying to my OP : http://www.dslreports.com/forum/r31310138-Insane-bufferbloat-after-recent-modem-update-Videotron

I'm unable to alleviate the download bufferbloat with SQM but am happy to experiment if you have ideas. As it stands, anytime someone downloads during a gaming session here, an avatar dies. It reminds me of someone picking up the phone during good old dial-up days. I'll try to get the modem replaced.

FWIW, I've been rather happy with my Arris SB6183, on a 300/30 link. I get weird microbursts of latency and such, but I'm pretty sure it's the cable service itself, rather than the modem. Been wrestling with them to try and get that worked on.

Too bad about the Puma 6 modem. I had read in that same thread rumors of the company working on a firmware fix, never showed up, but I haven't been following it lately.

You might try lowering your upload limit, could be you need a little more overhead. I've noticed having "enough" reduction in the upload can help the download, since there's always a little upload acking going on during a download.

Couple of questions for moeller0, in Sunspark's settings...

option linklayer_adaptation_mechanism 'default'
is it important to use "cake" as the mechanism, when you're using cake, or does it matter?

And, what's this "really, really, advanced" thing? Can you tell I don't get away from luci much?

option qdisc_really_really_advanced '1'

Well, "default" is just a way to tell sqm to use the cake method if cake is selected as qdisc and tc_stab otherwise as this seems to be the best thing to do. So qdisc=cake and linklayer_adaptation_mechanism 'cake' will have the same result as linklayer_adaptation_mechanism 'default'.

In luci-app-sqm there are a number of checkboxes with increasingly scary names that expose more dangerous options, qdisc_really_really_advanced is the one that exposes the advanced qdisc configuration strings, that are currently required to access cake's more recent options manually. BUT luci-app-sqm will only store those options to /etc/config/sqm if the checkboxes are checked... (if unchecked luci-app-sqm will actually erase those options from the config file).

Here is from the luci code:

ad2 = s:taboption("tab_qdisc", Flag, "qdisc_really_really_advanced", translate("Show and Use Dangerous Configuration. Dangerous options will only be used as long as this box is checked."))
ad2.default = false
ad2.rmempty = true
ad2:depends("qdisc_advanced", "1")

sqm-scripts proper does not care for this so you can avoid this in your config, but looking at the config via the luci app and saving will erase those options guarded by qdisc_really_really_advanced from the config file, which is a bit unexpected, hence the recommendation to also add this when manually editing /etc/config/sqm...

Hope that helps...

I have an Archer C7v2 with LEDE 17.01 running SQM "Piece of cake" without bufferbloat (A on dslreports).

I get A+ on Ethernet and A on WiFi.

This modem is not affected by the Puma6 issue since it's a Broadcom device.