Openwrt mt76 vs Unifi driver performance on Unifi 6 Lite APs

spm1001 · December 21, 2021, 10:55pm

Continuing the discussion from AQL and the ath10k is lovely - forum mods hopefully this thread is OK here but please move if not.

Thanks for your feedback @dtaht. Ultimately, per the title, I will run and post comparisons of the kind you are after, namely:

for stock Unifi drivers vs OpenWRT mt76
in situations of varying distance from the APs to wireless client
using both the Apple networkQuality and flent rrul
with ECN on and off at both ends

...but to begin with I was intrigued by your idea of doing a wired baseline to reason against, especially for the Apple test. That should show higher RPMs right?

Well, sort of... I did a few trials where the path was:

MacBook running networkQuality ->
Gigabit USB-C Ethernet adapter ->
Gigabit switch ->
Ubuntu 20 PC with gigabit NIC running the go version of networkQualityd (the my.local.server from previous post.)

As you can see, RPMs in this setup are high - 5647, 6037, 6357, 5217, 4549, 2222, 5052 - but variable and the upload/download flows the test decides to run aren't always the same.

% networkQuality -C https://my.local.server:443/config      
==== SUMMARY ====                                                                                         
Upload capacity: 353.619 Mbps
Download capacity: 379.572 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (5647 RPM)
% networkQuality -C https://my.local.server:443/config
==== SUMMARY ====                                                                                         
Upload capacity: 316.223 Mbps
Download capacity: 394.247 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (6037 RPM)
% networkQuality -C https://my.local.server:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 305.368 Mbps
Download capacity: 377.304 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (6357 RPM)
Base RTT: 1
Start: 21/12/2021, 21:44:28
End: 21/12/2021, 21:44:42

% networkQuality -C https://my.local.server:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 380.640 Mbps
Download capacity: 391.175 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (5217 RPM)
Base RTT: 1
Start: 21/12/2021, 21:44:51
End: 21/12/2021, 21:45:05

% networkQuality -C https://my.local.server:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 340.259 Mbps
Download capacity: 410.862 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (4549 RPM)
Base RTT: 1
Start: 21/12/2021, 21:45:12
End: 21/12/2021, 21:45:26

% networkQuality -C https://my.local.server:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 289.643 Mbps
Download capacity: 380.904 Mbps
Upload flows: 12
Download flows: 20
Responsiveness: High (2222 RPM)
Base RTT: 1
Start: 21/12/2021, 21:45:37
End: 21/12/2021, 21:45:51

% networkQuality -C https://my.local.server:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 359.702 Mbps
Download capacity: 359.302 Mbps
Upload flows: 12
Download flows: 12
Responsiveness: High (5052 RPM)
Base RTT: 1
Start: 21/12/2021, 21:46:05
End: 21/12/2021, 21:46:14

So then I wondered 'if two machines don't constitute a stable configuration, why not do it all on one'? In other words - have the client (networkQuality) and server (networkQualityd) running on the same machine?

It took a bit of mucking about, in particular adding 127.0.0.1 my.local.server.com to /etc/hosts to allow the SSL certificate for my.local.server.com to work, and compiling the Swift version of networkQualityd, to create (I think) the path:

MacBook running networkQuality ->
127.0.0.1:443 ->
Same MacBook running networkQualityd bound to loopback

And this path gives results that are perhaps more puzzling!

The download soars into the Gbps region (presumably as it's just going in and out of the network stack without touching actual NIC hardware), but the upload tops out at 100Mbps level which seems odd, and RPMs are middling at 1886, 1771, 1735, 1528.

% networkQuality -C https://my.local.server.com:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 103.213 Mbps
Download capacity: 10.131 Gbps
Upload flows: 12
Download flows: 12
Responsiveness: High (1886 RPM)
Base RTT: 1
Start: 21/12/2021, 22:03:21
End: 21/12/2021, 22:03:30

% networkQuality -C https://my.local.server.com:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 108.970 Mbps
Download capacity: 10.161 Gbps
Upload flows: 12
Download flows: 12
Responsiveness: High (1771 RPM)
Base RTT: 1
Start: 21/12/2021, 22:24:25
End: 21/12/2021, 22:24:34

% networkQuality -C https://my.local.server.com:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 111.286 Mbps
Download capacity: 10.056 Gbps
Upload flows: 12
Download flows: 12
Responsiveness: High (1735 RPM)
Base RTT: 1
Start: 21/12/2021, 22:24:49
End: 21/12/2021, 22:24:58

% networkQuality -C https://my.local.server.com:443/config -v
==== SUMMARY ====                                                                                         
Upload capacity: 94.539 Mbps
Download capacity: 9.442 Gbps
Upload flows: 12
Download flows: 12
Responsiveness: High (1528 RPM)
Base RTT: 1
Start: 21/12/2021, 22:25:08
End: 21/12/2021, 22:25:17

Furthermore, running networkQuality with the -s switch to do upload and download separately completes the download portion of the test but then gets a timeout error (presumably from the server process not responding.)

% networkQuality -C https://my.local.server.com:443/config -v -s
==== SUMMARY ====                                                                                         
Upload capacity: 0.000 bps
Download capacity: 10.267 Gbps
Upload flows: 0
Download flows: 12
Upload Responsiveness: Low (0 RPM)
Download Responsiveness: Low (0 RPM)
Base RTT: 1
Start: 21/12/2021, 22:27:01
End: 21/12/2021, 22:27:41

Error: Error Domain=NSURLErrorDomain Code=-1001 "The request timed out." UserInfo={_kCFStreamErrorCodeKey=-2103, _NSURLErrorFailingURLSessionTaskErrorKey=LocalUploadTask <79A2C95A-28F7-44FC-B03D-061148D759B4>.<1>, _NSURLErrorRelatedURLSessionTaskErrorKey=(
    "LocalUploadTask <79A2C95A-28F7-44FC-B03D-061148D759B4>.<1>"
), NSLocalizedDescription=The request timed out., NSErrorFailingURLStringKey=https://my.local.server.com:443/slurp, NSErrorFailingURLKey=https://my.local.server.com:443/slurp, _kCFStreamErrorDomainKey=4}

IIRC the networkQuality client does some probing before it makes decisions about how many streams to set running, and maybe its conclusions are susceptible to variation that early probing? Why does the number of streams chosen jump from 12 to 20? I can't find the networkQuality client source to investigate further - perhaps that's because it isn't released anywhere?

I'd also like to be able to run an rrul test on the same topology (i.e. having netserver and flent on the same machine), but I'm stuck as I can't seem to compile the earlier version of fping for MacOS that I need to avoid the error referenced here but even with the test above there's something odd/interesting going on I think.

Anyway, more to come as I mention above but wanted to post this for starters.

dtaht · December 22, 2021, 4:46pm

Good idea to fork this. Your localhost test is very interesting. Given the rise of containers, being able to transfer data well in both directions is something no-one to date has tried much!

namidairo · December 23, 2021, 1:31am

I did hear rumblings about the performance being a little lacklustre on the stock image, but I guess we'll see.

I would note that you should probably test throughput/latency/etc in both directions, both simultaneously and individually. Since I believe you may only have the wlan->ethernet offload from the radio but no ethernet->wireless encap offload? (I don't think the soc design had support for it at that stage?)

I'm not entirely sure you'd hit a CPU bottleneck there before hitting theoretical upper limits of a 2x2 11ax connection, but it would be a good thing to investigate regardless. (Since I believe you easily hit 400+ on 11ac models of similar design)

I'm not sure how you would test multiple client throughput in a controlled manner, but that could be something to look at too.

I suspect the flent graphs for anything done on the mt7603 radio may look like utter dog doo-doo though, even with airtime fairness advertised. The 2.4ghz portion of the mt7915d should be interesting data though.

spm1001 · December 23, 2021, 8:35am

Thanks - that’s helpful guidance - will incorporate that into the tests. You’ve obviously got a deep knowledge of these devices!

The 2.4ghz portion of the mt7915d should be interesting data though.

Didn’t realise there was 2.4 on the 7915. Happy to try that too but not sure how to set it up. On the Wireless LUCI page I see a listing for two radios but when I click through to the configuration for 7915 I can only choose a 5ghz channel. Is there something I can do in the CLI?

namidairo · December 23, 2021, 8:49am

Sorry to get your hopes up, I was assuming it was a DBDC card because it was 2x2... Please ignore.

dtaht · December 23, 2021, 6:12pm

I'd really like to know the output from their fq-codel implementation on successive different runs of the networkQuality tool. You can snag that via

netstat -qq -I en0 # or the correct device

My concern is that we hit "drop_overlimits" which means that you ran out of queue space.

Relative to that would also be a packet capture of a high performing run vs a less performing one. I was seeing tcp rsts in one set of data using a preliminary version of this tool.

dtaht · December 23, 2021, 7:53pm

This fellah is attempting a re-implementation of the responsiveness client. https://github.com/hawkinsw/goresponsiveness