Best practices and tools for measuring wifi performance

Installing and Using OpenWrt
3

[Reserved 2]

4

[Reserved 3]

5

Me to.

But you are seriously the first one to make three reserved posts in your own tread without anyone actually giving some feedback on your first hypothesis.

If you ask me “measuring perfection” is addictive for the human brain, once in that loop there will never be enough measuring and when the measuring to perfection is finally done. Then add another decimal and the carrousel starts all over.

So does the world change if everyone measures the wifi performance with all these tools? Does actually anyone have time to use the wifi if measuring it this exactly all the time?

1 Like
6

I recently wanted to post the systemlog and kernellog from OpenWRT, because i encountered an issue and i had to post those two in three different posts, because there is a characterlimit, so I wanted to make sure to have enough space :slight_smile: Maybe we will not need the reserved space here, but who knows.

You are free to fix your wifi without measurements. You are also free to not fix your wifi at all. With help of some measurements, my personal world changed though, because in my flat, i found a better location for my wifi-repeater. I am sure, somebody who actually knows how to code may make even better use of precise measurements. #Education. #Free Speech. #People have to start somewhere learning about things.

7

Did you post those logs as “preformated text”?

8

I used

```

```

as you can see here: OpenWrt Support for D-Link DAP-X1860 - #88 by ThiloteE

9

Maybe add crusader (https://github.com/Zoxc/crusader) as a nifty tool to generate load between two endpoints and also concurrent latency measurements. Not as powerful as i.e. iperf2 but it will generate simple graphs to allow assessing the measurement immediately after each run.

2 Likes
10

Another thought: Instead of attempting to use your router to source/sink data (which will affect its routing performance), you could use a small single-board computer with Ubuntu to run Flent, Crusader, iperf, etc.

I have a RPi 4 (purchased before the pandemic - so not too expensive) and an Odroid (don't remember what model) attached to Ethernet ports on my router...

Does anyone have recommendations/links for other (modest price) devices that could run those tests?

1 Like
11

Yeah, and even if you have a (spare) desktop computer (Linux/Mac/Windows) connected with gigabit Ethernet to the same network as the wifi access point, run iperf there as the other end. Ensure the Ethernet speed is faster than the highest possible speed of the wifi network.

Anti-pattern: measuring wifi by connecting to an upstream server. You'll be hard-pressed to segregate the wifi link peformance and the upstream link/general Internet's performance.

1 Like
13

Thank you for starting this great thread. A lot of good advice in here.

I also have a few tips to contribute.

iperf3 tips

One important thing to note here is that by default iperf3 UPLOADS data from the client to the server.

The reason this is important to consider on WiFi is that WiFi clients oftentimes have much weaker antennas than WiFi routers.

For example, on my M1 Macbook Air I can get up to 700-800 mbit when downloading from the server, but only 400-500 when uploading.

This can be solved by using this option:

-R, --reverse             run in reverse mode (server sends, client receives)

I would suggest to add that to all the commands in your post, because download is probably what most people are interested in testing.

Where you run the testing tool server and client is important

A lot of the performance is affected by the processing speed of the components between the client and the server.

For example, if you have a router with a weak CPU and run the iperf3 server on the router, then it's possible the speeds will be limited, because iperf3 itself uses up a large portion of the router's available CPU and starves the router of the CPU to achieve full wifi transmission rates.

The solution would be to connect a wired computer to the router that runs the iperf3 server. And then run the test from a wifi client to the wired computer.

But this stil doesn't give you the FULL picture. Because routing from lan to wan also requires CPU power and this is not being done when testing on lan only. This is especially important if you want to test the maximum bandwidth your router can handle with SQM.

If you want to determine the routing capabilities, then the iperf3 server should be "behind the WAN port" of the router.

Realistically you would need two routers for that setup.

For example, maybe you have the ISP router that's connected to the internet and has a LAN of 192.168.1.0/24.

Then you connect the new router you want to test to it that has its own lan of 192.168.2.0/24. Plug the new router's WAN port into the ISP router's LAN port so that it is in both networks and routes between them.

Then you connect a the iperf3 server to the ISP router. And you connect the wifi client to the new router.

Then you run the test from a client connected to the new router. This fully tests the performance of your router and all components involved.

Again, this is most likely not necessary if you just want to test your current network connection as you can run speedtests directly to the internet to test the routing performance.

But if you want to determine the maximum performance your router is capable off, then that's one way to do it.

Test bufferbloat on both wired and wifi

A similar issue si when you are measuring bufferbloat (latency under load). You should always first test it on ethernet to determine the bufferbloat performance of the routing component and SQM system.

This tests the bufferbloat between router and ISP.

After that you can test it on WiFi to determine if there is any additional bufferbloat between WiFi -> Router.

Avoid WiFi scanning tools while testing

Another important thing that could trip you up if you are testing a lot of things is WiFi scanning.

If you run a software that analyses channels, then the software is going to make periodic scans of available WiFi access points.

When it does that, it oftentimes interrupts the entire WiFi connection. This causes all WiFi transmissions to stop for a few miliseconds or even seconds.

This can cause you to have huge ping spikes and reduced transmission that will give you the wrong impression of your WiFi performance.

2 Likes
14

@ThiloteE You are only interested in English-language texts, I suppose?

I wrote something in German a while ago, because I was active in an action group who helped to bring fiber-optical internet to our small place, and in the aftermath I got countless questions re. WiFi problems, so I created a blog post about how to debug WiFi issues... I also created a similar blog post about internet issues in general (again in German).

In case you are interested anyway, please let me know, and I'll share the links here.

15

Sure, i would be delighted :slight_smile: although I think most people here speak English, as it seems to be the current lingua franca. I am from Germany too :wink:

1 Like
16

Yeah, right. But maybe for German people less fluent in English the posts could be helpful?

Anyway, here's the blog posts, feel free to include them in your collection (or to omit them):

(I suspected already you are German because of your username -- the leading part is a well-known German given name... :wink: )

2 Likes
17

Thanks for the time you spent and the info shared. I'm a user and seldom need to get down to the packet level. My Fave tool is android app "WiFiAnalyzer", the VREM version. You can watch your radio in almost real time. Handy for initial config of wrt3200acm which has too many radios.

1 Like
18

== Update 1 iperf3 under Windows on Arm
https://njit.io/kb/os/windows/how-to-compile-iperf3-for-windows-using-cygwin/

Followed the above post, installed Cygwin, compile iperf3 under Cygwin, run iperf3 under Cygwin, now I got same speed as running Speedtest in Edge browser. (Chrome Browser is much slower, due to X64 binary for Chrome only)

===
Where I can find out the best tools for the listed tests under:

Windows x64
Windows for Arm
Linux
iPhone
Android

I want to test with my Windows for Arm laptop, Android (Nokia 9) and iPhone 13, with Dynalink DL-WRX36 Stock / OpenWrt.

The Bufferbloat test speed is faster than iperf3 speed, so I suspect the Win64 binary is not running well under Windows for Arm.

19

Please open a new forum post for your issue. We can debug it over there. As I mentioned in my first post "This thread here is about teaching. This thread here is not about debugging your very personal setup of OpenWRT". When you open a new thread, I would ask you to explain your setup (where is the client, where is the server) and which commands you used for the measurements. Also, your exact configuration of OpenWRT can be important.

With regard to your questions:

  1. It depends on the specific tools, if they provide builds and releases for multiple operating systems. About "best" tools, I don't know, but Iperf3 should be a "good enough" tool for what you want to do.
  2. "The bufferbloat test is faster than iperf3 speed" depends on how and where you set up your measurements. Also, you should test wifi and the connection from your main router to your Internet service provider (ISP) separately. It could be the case, that your wifi is worse than the speed provided by your ISP. The bufferbloat test by waveform should be conducted from your main router or at least from a client connected to your main router via cable and not via wifi, as wifi uses buffers and may distort the bufferbloat measurements.
  3. Alternative download location for newest version of iperf3 for windows: https://community.chocolatey.org/packages/iperf3
20

Thank you for the reply.

I am sorry that I do not make myself clear enough.

From the Windows on ARMs laptop, the performance difference is due to the version of iperf3 used (running on that laptop).

Once I compiled iperf3 under Cygwin, the figure matches (iperf3 w/ Cygwin Vs Speedtest.net / Waveform Bufferfloat)

I am learning about the tools, and trying to find out which tool is available to which platform.

Like,

  • flent seems not available on Windows
  • TWAMP-GUI not available on Fedora

Until I learn to use more tools, iperf3 will be the tool I am testing for the moment.

21

flent itself is available on windows IIRC, but its current main data source netperf is not, so you can create plots under windows, but you can not actually take measurements.

So iperf3's biggest advantage over most of the rest of the list is that there exist some publicly open servers on the internet than can be used for some tests.
However, for real reproducible tests one probably needs to deploy one's own remote measurement points and then all tools are more or less available again...

1 Like
22

Sidenote: a lot of the measurement packages seem either "complete" or stalled in development...
Which is a real pity especially for the one-way delay measurement tools out there, just had a look at owping and e.g. the absence of an installable packet for ubuntu does not fill me with confidence (let alone OpenWrt)...

I wish IPv6 had mandated that ICMP timestamps would still be required (and I dream about a tighter definition/resolution of what to reply, so e.g. microseconds since midnight UTC instead of millisecond since midnight UTC and no backsies...). Alas, not much use closing that barn, as the time window to solve that is clearly well in the past by now.

1 Like
23

A command that queues 14 Iperf3 throughput tests and writes the output to a text file:

Today I learned how to queue commands in cmd.exe and how to write output to a textfile. Cmd.exe is the Windows 10 command line tool; The command should also work with Linux, but I have not tried it yet.

Here is my final command, which singlehandedly triggers **14 separate iperf3 throughput tests and writes the output to a textfile located in the folder the command was called from.

(echo TCP test && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 && echo. && echo. && echo TCP test with 2 client streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 && echo. && echo. && echo TCP test with 4 Client Streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 && echo. && echo. && echo TCP test with 8 Client Streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 && echo. && echo. && echo Bidirectional TCP test && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 --bidir && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 --bidir && echo. && echo. && echo Bidirectional TCP test with 2 Client Streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 --bidir && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 --bidir && echo. && echo. && echo Bidirectional TCP test with 4 Client Streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 --bidir && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 --bidir && echo. && echo. && echo Bidirectional TCP test with 8 Client Streams && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 --bidir && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 --bidir && echo. && echo. && echo TCP test in reverse && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 -R && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 60 -i 10 -R && echo. && echo. && echo TCP test with 2 Client Streams in reverse && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 -R && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 2 -t 60 -i 10 -R && echo. && echo. && echo TCP test with 4 Client Streams in reverse && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 -R && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 4 -t 60 -i 10 -R && echo. && echo. && echo TCP test with 8 Client Streams in reverse && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 -R && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -P 8 -t 60 -i 10 -R && echo. && echo. && echo UDP test && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 20 -i 2 -u -b 1000M && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 20 -i 2 -u -b 1000M && echo. && echo. && echo UDP test in reverse && echo iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 20 -i 2 -u -b 1000M -R && echo. && iperf3 -c IP-ADRESS-OF-IPERF-SERVER -p 5201 -t 20 -i 2 -u -b 1000M -R) | tee output.txt

What to do if you want to use this command:

On Iperf server side:

  1. Temporarily open relevant ports (5201) in the firewall, which are necessary for Iperf to work
  2. Iperf3 -s

On Iperf client side:

  1. copy 'n paste the command into a text editor
  2. search and replace IP-ADRESS-OF-IPERF-SERVER with the IP adress of your Iperf server.
  3. (Optional) search and replace 5201 with the port your Iperf server listens at. (5201 is Iperf3's default)
  4. (Optional) search and replace output.txt with the filename of your choice

The long command explained:

I use following syntax:

(echo. && echo. && echo <Iperf3command1> && echo. && <Iperf3command1> && echo <Iperf3command2> && echo. && <Iperf3command2> && echo <Iperf3command3> && echo. && <command3>...) | tee <filename>

The important bits (apart from iperf3 related) are ...

  • echo. = A new line. This will make it easier to find the tests you were looking for in the output file. Simply for Aesthetics.
  • echo <command> = The echo command in Linux is used to display a string provided by the user. The syntax is echo [option] [string] (More info here: https://phoenixnap.com/kb/echo-command-linux). What is does is to print the text you entered after it. With echo <command> && <command> it will have the command you entered precede the output. This will make it easier to find the tests you were looking for in the output file.
  • && = Example: command1 && command2 = Used to run the command following && only, if the command preceding the symbol is successful. Cmd.exe runs the first command, and then runs the second command only if the first command completed successfully.
  • <your-command> | tee result.txt = Show output in Windows cmd, but also send to output file (https://stackoverflow.com/questions/25702196/how-to-save-iperf-result-in-an-output-file). The () are necessary, when multiple commands are queued, otherwise only the last command before | tee will be written to the file.

What advantages brings command queuing?

You will save lots of time. Gone are the days when you have to wait in front of your computer for a test to finish, before you can start the next command. With this syntax you can trigger multiple tests with a single press of the enter button and then spend some time with friends or simply enjoy life and come back in 30 minutes and everything should be done by then :slight_smile:

1 Like