SQM / Cake Link Layer Adaptation

Installing and Using OpenWrt Network and Wireless Configuration
#1

Regarding Link Layer Adaptation, the documentation (https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details) states :

Set the Per-packet Overhead to 44 bytes if you use any kind of DSL/ADSL connection to the Internet (that is, if you get your internet service through a telephone line).

and:

It is especially important to set the Link Layer Adaptation on links that use ATM framing (almost all DSL/ADSL links do), because ATM adds five additional bytes of overhead to a 48-byte frame

and:

Please note that if your IS uses a mandatory VLAN on the bottleneck link, please add 4 additional bytes to the overhead to tell sqm to properly account for it.

Q: What should be the optimal value for a ATM / ADSL line with mandatory VLAN tagging? 48 ?

#2

That just means each data packet is split into sub-packets of 48 bytes as ATM uses so called cells which package 48 bytes of user data into 53 bytes by adding a 5 byte cell header (that is only visible/used on the ATM part of the link, so you will most likely never see this, but it eats into the available bandwidth. The encapsulation scheme AAL5 that is used for data packets over ATM additionally requires that each data packet is sent in an integer number of ATM cells so the last cell will also contain padding also eating into the available bandwidth.

The save bet would be 48 bytes, the likely bet 44 bytes and the "optimal" for your link is best deduced heuristically following the instructions in https://github.com/moeller0/ATM_overhead_detector

#3

Thank you.

Should I run the overhead detector with or without SQM/Cake enabled ? Or it does not matter?

#4

On a true ATM link it does not matter, as the test does not actually try to saturate your link at all, but rather it measures the ICMP RTT (ping) as a function of ICMP packet size. This actually works best if the network is not or only lightly loaded, but no need to disable SQM during the test.

1 Like
#5

Successfully generated 36 MB of ping data with the script... which I will try to analyze using Octave tomorrow.

EDIT: Worked like a charm, and here are the results.

pingsweep_results
pingsweep_results.png1200×900 19.2 KB

The command line outputs the following interesting message:

According to http://ace-host.stuart.id.au/russell/files/tc/tc-atm/
44 bytes overhead indicate
Connection: PPPoE, LLC/SNAP+FCS RFC-2684
Protocol (bytes): PPP (2), PPPoE (6), Ethernet Header (14), Ethernet PAD [8] (0)
, Ethernet Checksum (4), ATM LLC (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8
) : Total 44

OR

Connection: PPPoE, LLC/SNAP RFC-2684 + VLAN tag terminated at modem
Protocol (bytes): VLAN tag (4), PPP (2), PPPoE (6), Ethernet Header (14), ATM LL
C (3), ATM SNAP (5), ATM pad (2), ATM AAL5 SAR (8) : Total 44

Notre the second option, which includes VLAN tagging, but sums up to 44 bytes overhead.
As this theoretical explanation and the test both point to the same, 44 bytes it will stay.
Maybe the wiki should be updated?

1 Like
#6

Where and why? We could up the recommendation to 48, but as far as I can tell adding the ethernet frame-check sequence to the ATM payload really really is rare... so 44 is already taking a potential VLAN tag into account. 48 to my current knowledge is the absolute maximum, but I am sure one or the other ISP will come up with an even crazier scheme in the future :wink:

#7

What I mean is that 44 is working perfectly for me, using PPPoA + mandatory VLAN.
However, the wiki seemed to indicate that when mandatory VLAN tagging is present, one should add 4 extra bytes (ie increase to 48). This recommendation does not seem to be in-line with the reference above and also not inline with the results of my tests.

#8

Are you sure that this should not be PPPoE?

Which still is true, but only if based on either knowledge of the used encapsulation, or if one wants to be on the save side regarding the FCS.

Well, my recommendation is to run the collector and detector scripts :wink: (BTW, my ISP once transiently changed the encapsulation to be 4 bytes larger, only to switch back a bit later, I noticed as that changed the results of my flent rrul_cs8 bufferbloat tests noticeably and re-ran the collector/detector combo).

#9

It appears that it is PPPoA on the ISP side and PPPoE on the modem-router link. The Vigor modem takes care of that.

#10

That is rather a rare situation, which ISP are you using?

#11

How much does a couple of bytes really matter here. I'm thinking sure it's important to get close but unless you do a lot of small packet traffic, 5-10 bytes on top of 800-1500 byte traffic is a very small error. Even if you run say 2 VoIP calls and a game, you're talking less than a megabit of traffic, on modern connections that are 5-10 Mbps up at least again a small fraction of your traffic, so a 10% size error on a quantity that is 10% of packet size on traffic that is 10% of your bandwidth...

It might be best to just say 46 bytes and be done with it :slight_smile: the difference between even 30 and 60 is probably negligible to anyone not running a regression on a couple megabytes of data.

About the only people who should consider more precision is a call center running hundreds of VoIP calls on a dedicated line.

#12

Deutsche Telekom

#13

Well, I was under the impression, by reading the SQM docs, that getting this value "right" is essential for the proper functioning of Cake. "Right" may mean "Exactly right", i.e., not more, not less.

#14

I guess what I'm arguing is that exactly right is not needed, within 10% of the right value is probably just fine for all but a VOIP call center running hundreds of calls on a tight 10-15Mbps symmetric line.

The reason it's needed is to calculate the true packet size. If packet payload size is 1500 bytes then +-45 bytes makes only ~3% error. If packet size is 150 bytes like in a VoIP call, then 45 bytes is 33% error! So having the overhead included is important, but the difference between overhead say 44 and overhead 48 is 4 bytes and 4 bytes on 150 bytes is now back to 3% error so, do you know the capacity of your line to within 3% error? If not then even if you're a VoIP call center the error in your overhead if you just say 44 bytes is offset by the fact you aren't sure if you should put 10000 kbps or 9700kbps... If you're not a call center it's even an order of magnitude less important...

#15

I can guarantee that you just have plain PPPoE, otherwise the detected overhead would be different and DTAG only uses PPPoE :wink:

1 Like
#16

@moeller0 would it be possible to edit the wiki section on link layer adaptation to reflect the not too tight tolerances for most everyday purposes.

I agree that accounting for overhead is important, but I'm guessing we could add a section saying "Most people can just put 44 and it will work for you regardless of what your underlying technology is. For some people with fiber or cable connections, this may waste up to around 1-2% of your bandwidth, but it will bias you towards having lower bufferbloat rather than higher which is usually a good thing. The primary use case for more precise tuning is when your internet speed is relatively low (less than 5Mbps) and / or you have more than 20% of your internet speed in either direction taken up by small-packet traffic such as VOIP or gaming. If you have more than 5Mbps and/or you are not running a call center, further adjustment is probably not worth the effort and you should spend more time trying to better measure your reliable level of internet speed itself."

With that sort of preliminary warning, then we can still keep all the details for nerds like us but non-technical people will not feel like things are harder than they need to be.

:slight_smile:

Reading through the wiki, I'm seeing it's already intended to convey this kind of information, but maybe some rewording is in order to let people know more about what this stuff is supposed to be doing. I'm going to try to adjust it a little, feel free to revert my edits or further edit them if you disagree.

1 Like
#17

ATM is tricky in regards to bufferbloat, if you specify too much overhead you just loose some bandwidth, if you specify too little you will occasionally get an additional unaccounte-for almost empty ATM cell, eating 53 bytes of your sync bandwidth, with small packet, say VoIP ~200Byte packets that could mean 25% under accounting of the actual on-the-wire size. So cake would believe that it admits 200 byte to the wire, but really admits 253, which is a sure-fire way to get hard to debug packet-backspill distribution dependent buffer lost on your link. The problem here really is AAL5's requirement that user frames are packaged into an integer number of ATM cells.

#18

Yes, I definitely think the default recommendation should be to overestimate. Because of that I changed some of the wording on the wiki to recommend things slightly differently from what was there before. In particular for FTTP or metro Ethernet etc I said to choose ethernet with overhead and set 38 bytes. These connections tend to be fast, and the people reading the wiki are less likely to be doing call centers, so even if their actual overhead is 0, they will lose maybe 2.5 % bandwidth for large transfers, and they'll overestimate a 100kbps phone call by 15%, but 100kbps is less than 1% of a 10Mbps line, so unless they're running 10 simultaneous calls, again the "waste" is less than 1.5% for a typical metro E line, for a fiber at 100Mbps it's truly negligible unless they are running 1000 call simultaneous call center :wink:

#19

Not 100% sure whether these are your changes already, but the sqm entry page has something close to what you propose already:

In the Link Layer Adaptation tab, choose the kind of link you have:

For VDSL - Choose Ethernet, and set per packet overhead to 22
For DSL of any other type - Choose ATM, and set per packet overhead to 44
For Cable or other kinds of connections - Choose Ethernet, and set per packet overhead to 18
For true ethernet - Choose Ethernet, and set per packet overhead to 38

Click Save & Apply. That's it!

This will take probly not much more time to read than the text to explain why it does not really matter that much... Although I guess we should add:
If in doubt set this to 44.

#20

Here's the text from the details page with my edits, copied and pasted from: https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm-details#sqmlink_layer_adaptation_tab

SQM: Link Layer Adaptation Tab

The purpose of Link Layer Adaptation is to give the shaper more knowledge about the actual size of the packets so it can calculate how long packets will take to send. When the upstream ISP technology adds overhead to the packet, we should try to account for it. This primarily makes a big difference for traffic using small packets, like VOIP or gaming traffic. If a packet is only 150 bytes and say 44 bytes are added to it, then the packet is 29% larger than expected and so the shaper will be under-estimating the bandwidth used if it doesn't know about this overhead.

Getting this value exactly right is less important than getting it close, and over-estimating by a few bytes is generally better at keeping bufferbloat down than underestimating. With this in mind, to get started, set the Link Layer Adaptation options based on your connection to the Internet. The general rule for selecting the Link Layer Adaption is:

  1. Choose ATM: select for e.g. ADSL1, ADSL2, ADSL2+ and set the Per-packet Overhead to 44 bytes if you use any kind of DSL/ADSL connection to the Internet (that is, if you get your internet service through a telephone line).
  2. Choose Ethernet with overhead: select for e.g. VDSL2 and set the Per-packet Overhead to 22 if you know you have a VDSL2 connection (this is sometimes called Fiber to the Cabinet, for example in the UK). If you have a cable modem, you can try 22 bytes, or see the Ethernet with Overhead details below.
  3. Choose Ethernet with overhead if you have an actual Fiber to the Premises or Ethernet connection and set the Per-Packet Overhead to 38 bytes.
  4. Choose none (default) if you have some reason to not include overhead. All the other parameters will be ignored.

If you are not sure what kind of link you have, first try using Ethernet with Overhead and set 38 bytes. Then run the Quick Test for Bufferbloat. If the results are good, you’re done. Next, try the ATM choice, then the VDSL choice to see which performs best. If you have a slow connection such as less than 2Mbps in either direction and/or you regularly use several VOIP calls at once while gaming etc (so that more than 10 to 20% of your bandwidth is small packets) then it can be worth it to tune the overhead more carefully, see below for details.

Perhaps we can hammer out a reconciliation between this text and the text on the sqm (non-details) page, and make sure they both say the same useful thing.

1 Like