Best practices and tools for measuring wifi performance

Installing and Using OpenWrt
1

What is this thread about?

Having read a lot of forum threads, I came to the conclusion that many users (myself included) lack coherent information about how to conduct proper debugging and how to properly measure wifi performance. Furthermore, they lack information about tools available to do such. I invite everybody to point to proper practices, tools and methods in hopes of giving users and committers to OpenWRT alike a starting point to evaluate and debug their wifi issues. Posting measurements for educational purposes would be great. You are all very welcome to teach. Comparison of methods and tools sounds nice too, but might be a little exhaustive. Screenshots will be nice. I plan to sporadically update and edit this Initial message to account for responses and debate in this thread. Eventually, some info from here could be used for the docs.

What is this thread not about?

  • Expecting help, because your measurements point to flaws in OpenWRT code.
  • Debugging your very very personal setup of OpenWRT

→ If you have issues with your setup or if your measurements point to flaws in the code or hardware, please open a new forum post, instead of raising the issue here.

Best practices:

  • test latency (e.g. round trip time) simultaneously with throughput (show graphics side by side) [1]
  • run tests long enough [2]
  • test same configuration on same hardware with various distances/paths [2]
  • test same configuration on same hardware with various hindrances (e.g. walls inbetween)
  • test same configuration with multiple devices and different hardware
  • Measure latency under load ! [2]
  • Measure latency/throughput also with multiple (client) streams[2]
  • Do not interpret UDP and TCP results with equal weights [2]
  • Do not overvalue throughput rate over range without looking at latency or rate control [2]
  • Do not overvalue summary statistics that don’t look at inter-station behaviors [2]

Also check out the OpenWRT docs about Performance and logging

Sources: [1], [2]; OpenWRT Forum; Common Sense

Tools:

Simple methods for beginners:

For starters, usually it is advised to differentiate between your actual wifi and the connection from your main router to your internet service provider or some other endpoint on the internet.

Hosted tests on the web, for example for throughput like speedtest.net or fast.com will be bottlenecked by the slowest performing device along the way (or alternatively your contracted speedlimits, as offered by your internet provider). Tests for latency like the bufferbloat test hosted by waveform fall in the same category and best describe your connection from your router outwards towards the web.

By testing your wifi separately and restricted to your local network, as a major first step, you will be able to eliminate all sources of errors that stem from your internet provider and other connectivity issues along that path. Additionally, often, your wifi will be substantially faster than the speeds offered by your internet provider, but sometimes, the reverse can hold true as well. Also, for this reason, please test wifi separately from your outwards internet connection.

To find out what truly has an impact on your wifi connection, it is advised to test both throughput AND latency. Sometimes, your throughput will be great, but you will still face severe connectivity issues, because of latency. In the last section, a great variety of tools have been listed that (mostly) can be used to do exactly this. The abundance of tools may paralyse newcomers. Which tool is best? Which tool do I need? How can I best use them? - The answer to these question is difficult and may depend on your exact use case. It may also change from time to time, as many of these tools experience ongoing development.

To help you newcomers, I have listed a short introcution to Iperf3 (for throughput) and Wireshark (latency) down below, which should suffice for the most basic of needs and give you a rough idea about your devices current wifi capabilities:

1. Iperf3

  • How to capture throughput:

    • Run following command at the serverside: ĂŚperf3 -s

    • Run one of the following commands at the client side:

      TCP test, which measures the upload from client to server
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -t 60 -i 10

      TCP test with 4 Client Streams, which measures the uploads from clients to server
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 4 -t 60 -i 10

      TCP test with 8 Client Streams, which measures the uploads from clients to server
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 8 -t 60 -i 10

      TCP test; Reverse mode: server sends, client receives
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -t 60 -i 10 -R

      TCP test with 4 Client Streams; Reverse mode: server sends, clients receive
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 4 -t 60 -i 10 -R

      TCP test with 8 Client Streams; Reverse mode: server sends, clients receive
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 8 -t 60 -i 10 -R

      Bidirectional TCP test
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -t 60 -i 10 --bidir

      Bidirectional TCP test with 4 Client Streams
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 4 -t 60 -i 10 --bidir

      Bidirectional TCP test with 8 Client Streams
      iperf3 -c <IP of Iperf Server> -p <Port of Iperf Server> -P 8 -t 60 -i 10 --bidir

  • Tips and things to look out for when using Iperf3

2. Wireshark:

  • How to capture latency via testing round trip time:

    Use Iperf in combination with Wireshark. Capture traffic with wireshark while doing throughput tests with iperf, choose a random data packet from that time frame, then look at Wireshark's "Statistics → Stream graphs → Round Trip Time"

How to further debug wifi issues:

Additional info in German language:

Avoid configuration pitfalls:

  • Choose the best channel and band for your wifi. Avoid channels that are contested by neighbouring devices. The more devices are trying to use the same channel, the worse the connection, as there will be interference. To find the best channels, use WiFi Analyzer (VREM version)
  • Choose best wifi protocol for your use case:
    • 802.11n (wifi 4) at the 2GHz band, for long range (low throughput)
    • 802.11ac (wifi 5) at the 5 GHz band, for high throughput, but slightly lower range
    • 802.11ax (wifi 6) for higher throughput than n and ac and for less interference when multiple devices are in close proxity to each other and sharing channels/bands. Usable with both 2 GHz and 5 GHz. Theoretically estimated to have lower range in 2 GHz than n and in 5 GHz than ac.
    • Using VPN can drastically lower your experienced real world internet speed.
    • Many users report issues with SSID encryption in mixed mode. Avoid mixed encryption.
    • Have as less active devices connected as possible to avoid interference.
    • Avoid long distance and multiple walls.
    • Advanced configurations like roaming or mesh (802.11s, k, v...) bring their own problems and may be the cause of some of your network problems. If you really wanna know, disable these features during your tests
    • A wifi repeater (e.g. using relayd) by default halfs your wifi speed. If you can, use access point configuration and cable.

Miscellaneous:

Key Vocabulary:

Methods that look like, but that actually do NOT measure wifi performance:

Projects that try (or tried) to fix Linux Wifi

History, Videos and Literature about how wifi works, how it used to work in the past and historical attempts to fix it:

Dump of links, which are related to the topic, but that still have to be sifted through and that I am possibly considering to comment upon:

19 Likes
2

[Reserved]

3

[Reserved 2]

4

[Reserved 3]

5

Me to.

But you are seriously the first one to make three reserved posts in your own tread without anyone actually giving some feedback on your first hypothesis.

If you ask me “measuring perfection” is addictive for the human brain, once in that loop there will never be enough measuring and when the measuring to perfection is finally done. Then add another decimal and the carrousel starts all over.

So does the world change if everyone measures the wifi performance with all these tools? Does actually anyone have time to use the wifi if measuring it this exactly all the time?

1 Like
6

I recently wanted to post the systemlog and kernellog from OpenWRT, because i encountered an issue and i had to post those two in three different posts, because there is a characterlimit, so I wanted to make sure to have enough space :slight_smile: Maybe we will not need the reserved space here, but who knows.

You are free to fix your wifi without measurements. You are also free to not fix your wifi at all. With help of some measurements, my personal world changed though, because in my flat, i found a better location for my wifi-repeater. I am sure, somebody who actually knows how to code may make even better use of precise measurements. #Education. #Free Speech. #People have to start somewhere learning about things.

7

Did you post those logs as “preformated text”?

8

I used

```

```

as you can see here: OpenWrt Support for D-Link DAP-X1860 - #88 by ThiloteE

9

Maybe add crusader (https://github.com/Zoxc/crusader) as a nifty tool to generate load between two endpoints and also concurrent latency measurements. Not as powerful as i.e. iperf2 but it will generate simple graphs to allow assessing the measurement immediately after each run.

2 Likes
10

Another thought: Instead of attempting to use your router to source/sink data (which will affect its routing performance), you could use a small single-board computer with Ubuntu to run Flent, Crusader, iperf, etc.

I have a RPi 4 (purchased before the pandemic - so not too expensive) and an Odroid (don't remember what model) attached to Ethernet ports on my router...

Does anyone have recommendations/links for other (modest price) devices that could run those tests?

1 Like
11

Yeah, and even if you have a (spare) desktop computer (Linux/Mac/Windows) connected with gigabit Ethernet to the same network as the wifi access point, run iperf there as the other end. Ensure the Ethernet speed is faster than the highest possible speed of the wifi network.

Anti-pattern: measuring wifi by connecting to an upstream server. You'll be hard-pressed to segregate the wifi link peformance and the upstream link/general Internet's performance.

1 Like
13

Thank you for starting this great thread. A lot of good advice in here.

I also have a few tips to contribute.

iperf3 tips

One important thing to note here is that by default iperf3 UPLOADS data from the client to the server.

The reason this is important to consider on WiFi is that WiFi clients oftentimes have much weaker antennas than WiFi routers.

For example, on my M1 Macbook Air I can get up to 700-800 mbit when downloading from the server, but only 400-500 when uploading.

This can be solved by using this option:

-R, --reverse             run in reverse mode (server sends, client receives)

I would suggest to add that to all the commands in your post, because download is probably what most people are interested in testing.

Where you run the testing tool server and client is important

A lot of the performance is affected by the processing speed of the components between the client and the server.

For example, if you have a router with a weak CPU and run the iperf3 server on the router, then it's possible the speeds will be limited, because iperf3 itself uses up a large portion of the router's available CPU and starves the router of the CPU to achieve full wifi transmission rates.

The solution would be to connect a wired computer to the router that runs the iperf3 server. And then run the test from a wifi client to the wired computer.

But this stil doesn't give you the FULL picture. Because routing from lan to wan also requires CPU power and this is not being done when testing on lan only. This is especially important if you want to test the maximum bandwidth your router can handle with SQM.

If you want to determine the routing capabilities, then the iperf3 server should be "behind the WAN port" of the router.

Realistically you would need two routers for that setup.

For example, maybe you have the ISP router that's connected to the internet and has a LAN of 192.168.1.0/24.

Then you connect the new router you want to test to it that has its own lan of 192.168.2.0/24. Plug the new router's WAN port into the ISP router's LAN port so that it is in both networks and routes between them.

Then you connect a the iperf3 server to the ISP router. And you connect the wifi client to the new router.

Then you run the test from a client connected to the new router. This fully tests the performance of your router and all components involved.

Again, this is most likely not necessary if you just want to test your current network connection as you can run speedtests directly to the internet to test the routing performance.

But if you want to determine the maximum performance your router is capable off, then that's one way to do it.

Test bufferbloat on both wired and wifi

A similar issue si when you are measuring bufferbloat (latency under load). You should always first test it on ethernet to determine the bufferbloat performance of the routing component and SQM system.

This tests the bufferbloat between router and ISP.

After that you can test it on WiFi to determine if there is any additional bufferbloat between WiFi -> Router.

Avoid WiFi scanning tools while testing

Another important thing that could trip you up if you are testing a lot of things is WiFi scanning.

If you run a software that analyses channels, then the software is going to make periodic scans of available WiFi access points.

When it does that, it oftentimes interrupts the entire WiFi connection. This causes all WiFi transmissions to stop for a few miliseconds or even seconds.

This can cause you to have huge ping spikes and reduced transmission that will give you the wrong impression of your WiFi performance.

2 Likes
14

@ThiloteE You are only interested in English-language texts, I suppose?

I wrote something in German a while ago, because I was active in an action group who helped to bring fiber-optical internet to our small place, and in the aftermath I got countless questions re. WiFi problems, so I created a blog post about how to debug WiFi issues... I also created a similar blog post about internet issues in general (again in German).

In case you are interested anyway, please let me know, and I'll share the links here.

15

Sure, i would be delighted :slight_smile: although I think most people here speak English, as it seems to be the current lingua franca. I am from Germany too :wink:

1 Like
16

Yeah, right. But maybe for German people less fluent in English the posts could be helpful?

Anyway, here's the blog posts, feel free to include them in your collection (or to omit them):

(I suspected already you are German because of your username -- the leading part is a well-known German given name... :wink: )

2 Likes
17

Thanks for the time you spent and the info shared. I'm a user and seldom need to get down to the packet level. My Fave tool is android app "WiFiAnalyzer", the VREM version. You can watch your radio in almost real time. Handy for initial config of wrt3200acm which has too many radios.

1 Like
18

== Update 1 iperf3 under Windows on Arm
https://njit.io/kb/os/windows/how-to-compile-iperf3-for-windows-using-cygwin/

Followed the above post, installed Cygwin, compile iperf3 under Cygwin, run iperf3 under Cygwin, now I got same speed as running Speedtest in Edge browser. (Chrome Browser is much slower, due to X64 binary for Chrome only)

===
Where I can find out the best tools for the listed tests under:

Windows x64
Windows for Arm
Linux
iPhone
Android

I want to test with my Windows for Arm laptop, Android (Nokia 9) and iPhone 13, with Dynalink DL-WRX36 Stock / OpenWrt.

The Bufferbloat test speed is faster than iperf3 speed, so I suspect the Win64 binary is not running well under Windows for Arm.

19

Please open a new forum post for your issue. We can debug it over there. As I mentioned in my first post "This thread here is about teaching. This thread here is not about debugging your very personal setup of OpenWRT". When you open a new thread, I would ask you to explain your setup (where is the client, where is the server) and which commands you used for the measurements. Also, your exact configuration of OpenWRT can be important.

With regard to your questions:

  1. It depends on the specific tools, if they provide builds and releases for multiple operating systems. About "best" tools, I don't know, but Iperf3 should be a "good enough" tool for what you want to do.
  2. "The bufferbloat test is faster than iperf3 speed" depends on how and where you set up your measurements. Also, you should test wifi and the connection from your main router to your Internet service provider (ISP) separately. It could be the case, that your wifi is worse than the speed provided by your ISP. The bufferbloat test by waveform should be conducted from your main router or at least from a client connected to your main router via cable and not via wifi, as wifi uses buffers and may distort the bufferbloat measurements.
  3. Alternative download location for newest version of iperf3 for windows: https://community.chocolatey.org/packages/iperf3
20

Thank you for the reply.

I am sorry that I do not make myself clear enough.

From the Windows on ARMs laptop, the performance difference is due to the version of iperf3 used (running on that laptop).

Once I compiled iperf3 under Cygwin, the figure matches (iperf3 w/ Cygwin Vs Speedtest.net / Waveform Bufferfloat)

I am learning about the tools, and trying to find out which tool is available to which platform.

Like,

  • flent seems not available on Windows
  • TWAMP-GUI not available on Fedora

Until I learn to use more tools, iperf3 will be the tool I am testing for the moment.

21

flent itself is available on windows IIRC, but its current main data source netperf is not, so you can create plots under windows, but you can not actually take measurements.

So iperf3's biggest advantage over most of the rest of the list is that there exist some publicly open servers on the internet than can be used for some tests.
However, for real reproducible tests one probably needs to deploy one's own remote measurement points and then all tools are more or less available again...

1 Like