Also it would be useful to note that some TCP congestion algos do not coexist with ECN. As far as I remember (at least BBR) treats it as if no congestion had happened.
SQM? I must be missing something here, if ECN does not apply to forwarded packets....
BBRv1 indeed ignored ECN, but did not make sure it was not negotiated... BBRv2/3 learned the DCTCP dialect of interpreting CE marks. That is a bad fit for SQM's rfc3168 style ECN, but then BBR sort of ameliorates that issue by actively trying to share the capacity reasonably.
It was negotiated according to ss -tio output.
Still had no time to play with BBRv3.
I tried to explain the sequence of things on the numerated list on the first post, but obviously did not succeed. SQM will set the CE codepoint in the 2bit ECN bitfield of the IP header (both IPv4 and IPv6) on congestion, and it will do so independent on how the router's own TCP stack is configured. But it will do so only if the end points set either the ECT(1) or the ECT(0) codepoint in the IP packets to begin with. And typically they only do this when negotiating ECN use in a protocol like TCP or QUIC.
And you are right for ECN to be useful you need both the bottleneck AQM actually supporting IP-level ECN/CE marking, and the endpoints negotiating protocol level ECN and setting the appropriate IP header code points.
I guess one would need to disable ECN on host using BBR... That said, I am not Google (obviously) and hence have different constraints. For me default CUBIC with SQM resulted in decent throughput and decent latency under load (for non-capacity seeking flows). But then, as a hobbyist I do care much less than Google and co. how energy efficient/costly that ends up being...
Some people get confused that SQM only applies to WAN traffic and you still want ECN in you LAN (sure, if you want it enabled globally).
I think the best is not use BBRv1 at all.
For ECN to do any good you need an ECN enabled bottleneck... typical home networks either use a switch or WiFi for that. Home switches as far as I can tell do not support ECN (datacenter switches do often support dctcp-style ECN, but I have no idea about prices and availability). APs in theory could learn ECN or might already support it. @dtaht as part of make-wifi-fast did you try ECN on WiFi, and if so what was the conclusion?
The only thing I changed after playing a lot with TCP CC was in loading tcp_lp and setting peer congestion control in Transmission to lp. UDP connections are already on some sort of LEDBAT variant.
1 Like
Does it really not generate ECN signals when you pass data from on port of, let's say, Mediatek-based switch to another port on ECN-enabled OpenWrt?
This is more of a question about default behaviour of that unsupported (?) HQoS in some chips.
Not as far as I understand (so not all that deeply), for marking you need an active marking entity at the right place... you can in theory drive the marking from different sources (like measured congestion/queue length, or fraction capacity used, or ...) but you still need an entity on a port that actually measures that information and translate it into marks somehow.
Is it not hardware implemented? Like pause frames.
Dropping 10% of syn packets would not be acceptable at random.
Once the queue is full 100% of additionally ingressing packets will getvdropped, SYN packets or otherwise, 
RFC3168 ECN is enabled in the make-wifi-fast wifi code, but it is crippled by a bug I have been trying to rip out for 8 years now, where it is disabled if more than 4 stations are present on the link, which is the case for most wifi links. The patch fixing that is floating around in various openwrt custom firmwares.
I used then and still use ecn as a means of debugging network stacks: https://blog.cerowrt.org/post/crypto_fq_bug/
the linux fq_codel qdisc fully supports RFC3168 ECN marking by default, and it is still surprising to me how little it is used outside the apple ecosystem. It (and vastly reduced codel target) are a good fit for containers for example.
2 Likes
@moeller0 && @dtaht -- What about the setting net.inet.tcp.accurate_ecn as noted in this presentation?
Should it not also be set to 1 (enabled) for Apple/Mac devices?
Also, it should be noted that the OSX settings defined here can no longer be placed in /etc/sysctl.conf in newer MacOS versions (MacOS 10.9+). One must now define a LaunchDaemon (as something like /Library/LaunchDaemons/sysctl.plist) to apply the settings upon boot.
e.g.:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>sysctl</string>
<key>ProgramArguments</key>
<array>
<string>/usr/sbin/sysctl</string>
<string>net.inet.tcp.ecn_initiate_out=1</string>
<string>net.inet.tcp.ecn_negotiate_in=1</string>
<string>net.inet.tcp.disable_tcp_heuristics=1</string>
<string>net.inet.tcp.accurate_ecn=1</string>
</array>
<key>RunAtLoad</key>
<true/>
</dict>
</plist>
So rfc3168 ECN does not need this... that said more information is better than less information, so accurate ECN by itself might not hurt.
I am on record as not being a fan of the rfc3168 alternative rfc9331 (that requires something like accurate ECN), but I try to not delve too deep into this dislike in this thread as I want to keep it positive....
Fair enough 
I'm not up on the political sides of these things, but I did want to ask it here in case that setting did hold some significant value in configuring ECN.
Now, with all this said, I have had ECN enabled on several of my devices for some time now and I still show "zeros" for ECN stats when I run a netstat -sp tcp on my MacBook. Not sure what I've missed in the equation, so to speak.
I get all zeros when I try netstat instead of sudo netstat but i guess you tried that already?