Makes sense!!! I've got my macOS firewall always off at home. 
Thanks for the update. And interestingly enough my fping just works, looks like I've got a little bit of luck.
@dtaht: I ran Flent on the Macbook Air sitting in India over VNC. I tried netperf-eu.bufferbloat.net and singapore.starlink.taht.net .
Router/AP: Belkin RT3200 running OpenWrt 22.03.0-rc5
ISP: Airtel India, GPON Fiber, PPPoE, 40/40 Mbps Plan, IPv4 + IPv6
Location: (near) Chennai, Tamil Nadu, India
SQM on WAN: CAKE + layer_cake.qos, Download 42500, Upload 47500, Ingress options "ingress besteffort nat dual-dsthost", Egress options "egress diffserv4 nat dual-srchost", Ethernet with Overhead, Per Packet Overhead 44 bytes
WIFI Settings: 5 GHz, Band 36, VHT20, WPA2-PSK AES, 802.11w disabled, Beacon Interval 100, DTIM Interval 5
Client: 2017 Macbook Air (Intel x86_64)
OS: macOS 12.4 Monterey
WiFi Card: Broadcom BCM43xx
TESTED OVER VNC
Did you click the "Ok" button on the "Firewall Options" window after enabling "Enable Stealth Mode" ? The option doesn't take effect until you click "Ok".
in this test you seem to have your download sqm settings way too high for the perceived speed of this link (which looks like 16Mbit + whatever the cost of vnc), but please retry against my mumbai server.
Oh. I just checked with my relative and the Macbook was sitting in bedroom in one end of the house and the router is in living room in almost diagonally opposite end. Somehow 5 GHz signal still shows up across Brick wall.
If in that case the wifi was the bottleneck, the fq_codel aqm should have kicked in. It didn't. I would still be inclined to
doubt the isp's actual delivery of bandwidth outside their network, first, wifi, second.
Client in same room as router, no wall in between. VNC is also more responsive now. Fast.com measured 38/38 Mbps.
@amteza in neither case was ecn requested to negotiate. Apple is planning to make a major switch to their ecn support, so it wouldn't surprise me if they had disabled RFC3168 ECN entirely in some point release or another. It's been a pretty rare option. I happen to use it as a means to debug the AQM - seeing marks AND drops means there are problems elsewhere in the stack. Ah, well.
Both the up and download captures look solid to my eye.
@dtaht Did you mean to reply to me or to @amteza ? Anyway I also ran the Flent test to Singapore server.
Yes: from mumbai andsingapore, the SQM is controlling the link and barely touching the wifi. At one level that's good. At another level, your prior result through the wall indicates that we are not successfully AQM'ing a poor quality wifi link. Or that exiting the country to do the test behaves really badly.
update: ok, we have a wifi issue.
a tcp_ndown and tcp_nup test from mumbai whilst the laptop is on the wrong side of the wall would be helpful. I can imagine a tcp_nup test being the source of the 600ms latency observed. (I admittedly would be very happy if it was the nup test that was the source of most of the rrul latency on the wifi through the wall, because then it's buried in the OSX stack, not the openwrt stack)
I've got a pretty dark heart, but I feel like doing it today. 
A quick view of the rrul_be graph on 22.02.1:
Here you go:
Update: All files are wrongly named 22.02.1, they are 21.02.1, my bad.
@amteza the tcp rtt for the patches under test looks very solid here:
BTW, you can get plots like this via
tcptrace -G the.cap
And then looking for the largest *.xpl
xplot.org XtoY_rtt.xpl - the _tsg file is also useful.
These only work in one direction, so YtoX if you don't see anything sane,
Very few people know what a good result is supposed to look like in these ancient tools.
PS at least on linux (maybe not OSX?) tcpdump with -s 128 cuts the capture size to just what is needed.
1 Like
I don't think there's any OpenWrt version 22.02.1 . Did you mean 21.02.1 or 22.03.0-rcX ?
Do you know what, I will try a few runs in a QUEMU Linux ARM64 VM to see how it goes. But that will be tomorrow, no more time today for this kind of tests.
Yes, sorry about that.
@dtaht, just now on 21.02.1:
==== SUMMARY ====
Upload capacity: 34.483 Mbps
Download capacity: 237.941 Mbps
Upload flows: 16
Download flows: 12
Responsiveness: High (3622 RPM)
Base RTT: 11
Start: 11/7/2022, 1:25:24 pm
End: 11/7/2022, 1:25:37 pm
OS Version: Version 12.4 (Build 21F79)
@amteza thx for the baseline and going the extra mile for me today. Your 21.02.1 result is clearly better at T+120 forward, with less periodicity of the latency swings vs a vs airtime.
I can imagine prior to T+120 other sources of environmental noise, and osx issues also.
My fantasy is that everyone runs rrul tests all night long against the current patchset, and it doesn't crash and the noise is exposed as just that, wireless noise.....
1 Like
Happy to do that for you tonight with lates rc5 withour remove patches 334 and 335 if you wish. I will try to do it in a Linux ARM64 QEMU VM.
1 Like
RPM 3600 w 12 flows. That's awesome.
and 800 with w 12 flows on the latest patchset?
ok, well, trying --te=download_streams=4 with a capture next? I'm really haunted by that fq bug but the behavior here doesn't look like that.
not sure how a vm helps in testing wifi.