So you can add commands to /etc/rc.local. This script is executed once per boot, for things that are ephemeral like network interfaces (which can come and go, especially interfaces like pppoe-wan) you actually want to hook your commands into the hotplug mechanism so it is executed every time the interface comes up. In sqm-scripts case we have unchecked option fields exposed in the GUI in which you can put a lot of individual configuration options that are properly handled with hotplug.
Thanks Sebastian... (it is Sebastian, yes?)
I had already been running steps 1, 2 and 3. A few more questions if I may.
General question, If you setup a SQM, and don't have the SQM checkbox enabled, is there some part still working? This is how I go back and forth while testing, vs uninstalling luci-app-sqm.
For step 4, and further, do I want to play with the defaults for max size, number of entries, etc that I see there? Or just leave them with the defaults?
I'm using cake as my qdsc, will set it to cake for step 5. I have some question, (rusty networking knowlege) on whether I have larger than 1500 MTU. Is the max length from a tc -s qdisc the same as MTU, or not?
Will try the setting suggestions in 6.2 as well. It will be interesting to see how much CPU/IRQ overhead this adds, since my situation is also a case of how far can you push a Archer C7, with a 300/30mbit link. Something I'm also testing the limits of... but that's another thread... 
You can have multiple concurrent configuration sections and only those where the "Enable this SQM instance." checkbox is checked will be active, so you can have disabled sections in your configuration without any side effects on the active sections.
In all likelihood not, currently for fq_codel on most links I would recommend to set tcMPU to 64, but in real life, unless you flood your link with tiny packets you will not be able to see a difference.
For cake none of these values have any influence currently, in the future sqm-scripts will pass tcMPU to cake. And in essence the max_len you see is correlated for what you should put into the MTU field. But the relationship is a bit complex (as in it depends on a number of conditions), so unless your max_len/max_packet is larger than 2000 AND ypu are using fq_codel leaving these alone is fine.
Good point, please note that cake will tend to keep the configured bandwidth if CPU starved (while latency under load will increase a bit), while HTB+fq_codel will tend to keep the configured latency while bandwidth decreases. At 300/30 I would recommend to test both cake/piece-of_cake and fq_codel/simplest.qos as I assume the C7 will struggle to keep up with 300/30 and you will have to make a judgement call/policy decision...
Best Regards
It does. One of my questions there, is what happens when you do run out of CPU, how bad is it? Good to learn a bit about that. I had determined that I can run up to 140-145mbit configured bandwidth, and then I start bumping against 0% idle, and 100% sirq during a DSLReport run. I see maybe 75-85% CPU on the ksoftirq task. Don't know which of these numbers are the more important ones to judge overall loading with.
It will run higher than that, much more tightly pegged out. Seems to still be improving latency, but not as much.
From my observations the idle number is the most relevant, once you reach 0 shaping is going to suffer.
Well, as I said, cake will try to shape up to the configured bandwidth accepting more latency under load, while HTB+fq_codel will keep the latency close to theoretical limits while sacrificing bandwidth. How much each of the compromises the "other" measure really depends on how much you run them out of their comfort zone. I have no numbers though to quantify this effect....
Hi mindwolf,
July 24
My theoretical calculation. Please chime in with opinions and your experiences using this calculation.
Docsis 3.0 uses up to 42,880 Mbps per channel for a total of 686,880 Mbps (608,000 goodput) Ds and 122,880 (108,000 goodput) Us.
This might or might not be true (no time to look it up), but I assume that this is not going to be the relevant limit for you, as DOCSIS systems employ a traffic shaper that keeps the user traffic within the contracted rates. (Now if you have a DOCSIS segment all to yourselves that might be different, but in that case I am certain you need to already know more about DOCSIS than I ever will, so in that case just ignore me).
10% of the max achievable Ds and 5% of the max achievable Us. MTU 1500 minus 20 bytes for TCP overhead, 20 bytes of IPV4 overhead. 44 bytes total is = 6 bytes of overhead for Docsis 3.0, 14 bytes layer 1 overhead, 14 bytes layer 2 overhead,
This is a bit opaque for me, but the DOCSIS shaper is known to talke a full ethernet frame including the frame check sequence into account, so that will be 1518 Bytes at most. Now what layer that actually lives in is somewhat tricky for me to doiscern, as DOCSIS will drag in its own additional overheads (like the MPEG headers) but will leave out typical ethernet overhead like preamble and the inter packet gap... But for end users as far as I can see the ISPs shaper setting and the assumption that there will be 18 Bytes of overhead on top of the IP packet should hold true.
So...
686,880-10%=68,880
Not sure that I agree with that:
686880 * 0.9 = 618192 (686880 minus 10% of 686880)
686880 * 0.1 = 68688 (10% of 686880)
68,688/1,46044=2,070
68688-2070= 66618
122,880-5%=6144
6144/146044=185
6144-185=5959
Package listed as 60 Ds; 4 Us
The Above answers of 66618 & 5959 match very closely within 1% of my max achievable speeds without congestion on the wire without SQM enabled.
But more relevant, I would start from the measured goodput (in Kbps?) and calculate backwards from that (assuming TCP/IPv4):
66618 * ((1500 + 14 + 4) / (1500 - 20 -20)) = 69264.4684932 Kbps
5959 * ((1500 + 14 + 4) / (1500 - 20 -20)) = 6195.72739726 Kbps
So as typical for cable ISPs you seem to have more bandwidth allotted than you could expect from your plan. But if your ISP uses power-boost that bandwidth might not actually be sustained but might throttle back after the 10 seconds a speedtest typically runs*. Also due to the interaction of the DOCSIS shaper assumption of 18 bytes per-packet overhead in combination with the true DOCSIS per-packet overheads that extra bandwidth might just be a stopgap measure to make sure you get your contracted bandwidth even when you use smaller packets, but I have no data to back this up...
I offer all of this as simple opinion, as I have not fully understood what your exact argument is (as I am not a native english speaker, please forgive my denseness).
Best Regards
*) This is why I recommend to use a speedtest that can run for more than 10 seconds, see [SQM/QOS] Recommended settings for the dslreports speedtest (bufferbloat testing)
I found some good information from this link http://www.bowe.id.au/michael/isp/DOCSIS/collected-references/docsisdsusspeedplaybook8-5-16-160807165905.pdf
1 Like
Great find! Thanks for posting, even though this looks more like something for your DOCSIS-ISP as there is very little actionable stuff in this presentation for end users (and by very little I mean nothing), but certainly a good reference for DOCSIS issues.
Thanks! I have contacted the volpefirm.com, which seems to a VERY knowledgable source on DOCSIS related material. Waiting on a reply to clear up any confusion and hopefully with a straightforward answer.
1 Like
Sure, good idea to inquire with experts, but what exactly are you looking for? The 18 bytes overhead for DOCSIS comes straight from the DOCSIS standards documents:
Interestingly the shaper used in DOCSIS systems that limits a user's maximal bandwidth does completely ignore DOCSIS overhead and only includes ethernet frames including their frame check sequence (FCS 4 Byte). To cite the relevant section from the Docsis standard (http://www.cablelabs.com/specification/docsis-3-0-mac-and-upper-layer-protocols-interface-specification/):
"C.2.2.7.2 Maximum Sustained Traffic Rate 632 This parameter is the rate parameter R of a token-bucket-based rate limit for packets. R is expressed in bits per second, and MUST take into account all MAC frame data PDU of the Service Flow from the byte following the MAC header HCS to the end of the CRC, including every PDU in the case of a Concatenated MAC Frame. This parameter is applied after Payload Header Suppression; it does not include the bytes suppressed for PHS. The number of bytes forwarded (in bytes) is limited during any time interval T by Max(T), as described in the expression: Max(T) = T * (R / 8) + B, (1) where the parameter B (in bytes) is the Maximum Traffic Burst Configuration Setting (refer to Annex C.2.2.7.3). NOTE: This parameter does not limit the instantaneous rate of the Service Flow. The specific algorithm for enforcing this parameter is not mandated here. Any implementation which satisfies the above equation is conformant. In particular, the granularity of enforcement and the minimum implemented value of this parameter are vendor specific. The CMTS SHOULD support a granularity of at most 100 kbps. The CM SHOULD support a granularity of at most 100 kbps. NOTE: If this parameter is omitted or set to zero, then there is no explicitly-enforced traffic rate maximum. This field specifies only a bound, not a guarantee that this rate is available."
So in essence DOCSIS users need to only account for 18 Bytes of ethernet overhead in both ingress and egress directions under non-congested conditions.
This is not intended to curtail your research into the intricacies of the DOCSIS standards, just to state that as far as we know from sqm-scripts perspective 18 bytes overhead is the relevant number. BUT the trickier thing is to get the exact number the DOCSIS shaper is set to (luckily most cable ISPs seem to provision more gross bandwidth than visible from the contract, so using the contracted numbers should work as a starting point).
1 Like
if my router doesn't have nat I can use mpu 64 dual-dsthost y mpu 64 dual-srchost ?
Not sure what you mean here?
Probably, but you can always test wether it does the right thing:
A) On two computers run 1 speedtest each at the same time (use something like fast.com that can be configured for a relatively ;ong run time, so starting multiple instances get easier).
With or without the dual-xxxhost options you should see roughly the same rate for each speedtest
B) On the first computer start one speedtest like above, and on the second computer start two speedtests (in two different browsers probably).
Now you either get identical results for all three speedtests idicating that dual-xxxhost did not do its thing or roughly: X Mbps on the first Computer and X/2 on each of the speedtest on the second computer which would show dial-xxxhost working as intended.
Quick question though: Why post this on a thread about Link Layer Adaptation instead of starting a new focussed thread with an appropriate title?
to save time also what I am looking for is in this topic
When i use mpu 64 nat dual-dsthost the bandwidth is reduced because I think the problem here is NAT
In a public forum like this, it is always a good idea to look at "time-saving" from two perspectives:
A) What is the fastest way to get an answer
B) What way will be easiest/fastest to find by others that might stumble over the same issue and would be helped if they could find that post/thread easily.
Could you post speedtest results with and without that option, together with the matching output of tc -s qdisc?
Might well, be, but I still have not understood what the exact problem is to begin with, maybe you can help with describing your set-up and why you think that nat is problematic for you?
root@OpenWrt:~# tc -s qdisc
qdisc noqueue 0: dev lo root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8048: dev eth0 root refcnt 6 bandwidth 23Mbit besteffort dual-srchost nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 148268 bytes 927 pkt (dropped 0, overlimits 101 requeues 0)
backlog 0b 0p requeues 0
memory used: 10368b of 4Mb
capacity estimate: 23Mbit
min/max network layer size: 28 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Tin 0
thresh 23Mbit
target 5ms
interval 100ms
pk_delay 796us
av_delay 46us
sp_delay 2us
backlog 0b
pkts 927
bytes 148268
way_inds 0
way_miss 89
way_cols 0
drops 0
marks 0
ack_drop 0
sp_flows 5
bk_flows 1
un_flows 0
max_len 6056
quantum 701
qdisc ingress ffff: dev eth0 parent ffff:fff1 ----------------
Sent 293346 bytes 1646 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc fq_codel 0: dev eth1 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 7784797232 bytes 6251436 pkt (dropped 0, overlimits 0 requeues 1)
backlog 0b 0p requeues 1
maxpacket 1514 drop_overlimit 0 new_flow_count 8 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc fq_codel 0: dev eth2 root refcnt 2 limit 10240p flows 1024 quantum 1514 ta rget 5ms interval 100ms memory_limit 4Mb ecn drop_batch 64
Sent 1388475424 bytes 1549819 pkt (dropped 1, overlimits 0 requeues 5)
backlog 0b 0p requeues 5
maxpacket 1514 drop_overlimit 0 new_flow_count 9 ecn_mark 0
new_flows_len 0 old_flows_len 0
qdisc noqueue 0: dev br-lan root refcnt 2
Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0)
backlog 0b 0p requeues 0
qdisc cake 8049: dev ifb4eth0 root refcnt 2 bandwidth 95Mbit besteffort dual-dst host nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 18 mpu 64
Sent 316782 bytes 1646 pkt (dropped 0, overlimits 85 requeues 0)
backlog 0b 0p requeues 0
memory used: 13760b of 4750000b
capacity estimate: 95Mbit
min/max network layer size: 46 / 1500
min/max overhead-adjusted size: 64 / 1518
average network hdr offset: 14
Tin 0
thresh 95Mbit
target 5ms
interval 100ms
pk_delay 27us
av_delay 4us
sp_delay 3us
backlog 0b
pkts 1646
bytes 316782
way_inds 0
way_miss 91
way_cols 0
drops 0
marks 0
ack_drop 0
sp_flows 2
bk_flows 1
un_flows 0
max_len 2892
quantum 1514
root@OpenWrt:~#
because as a router I have rpi4 and I think it does not have NAT
I thought that removing nat and using mpu 64 dual-dsthost it could be the same
That might be, I am not sure what the nat keyword does when conntrack is not active though. I presumed that it would just operate on the IP addresses it does see. So, can you please run the test I proposed above and report the outcome here (preferably run the test twice, once with and once without the nat keyword).
I've long used SQM cake on my WRT32X (500Mbit cable modem) and it works great, A+ bufferbloat/A+ quality on dslreports.com/speedtest while maxing upload/download at 97% bandwidth (3% reserved). I used to set the overhead to 18 as per the man page:
Source: https://man7.org/linux/man-pages/man8/tc-cake.8.html
But recently I don't even bother setting overhead, I get the same results not touching that config, I think it recieves the overhead from the kernel it says in the man page. It would be nice to know exactly what to enter to optimize this.