iPerf3 - Understanding Results

I'm trying to get a better idea of performance across my network using iPerf3 but don't understand why I'm getting significantly different results for the same test. I wonder if anyone could enlighten me or suggest a way to identify the issue (if there is one).

My OpenWrt solution is running on a D-Link DIR 2660 with multiple VLANs configured. My main VLAN (VLAN10) is the HOME network ( The main OpenWrt interface for management is at

If I run iPerf3 between my main PC ( and a Rpi4 running iPerf3 on the same unmanaged switch I get 936Mbits/sec and the result is fairly consistent. When I connect that unmanaged switch to VLAN10 on OpenWrt and run the test again but iPerf3 server on OpenWrt I primarily get one of two results......either approx. 710Mbits/sec or 550Mbits/sec and again these two results are fairly consistent.

I realize the result will be slower than just running it on an unmanaged switch but what I don't get is why on one test I get 710Mbits/sec and then on the next test I get 550Mbits/sec.....the figures are always generally the same.

My first though was maybe it's to do with cpu utilisaton when I run the test or system load but I've configured "collectd" and the system is basically doing nothing when I run the tests (I've posted a screenshot which shows normal state and also when I have run iPerf3)

main PC = Windows10
iPerf3 command = iperf3.exe -c -t 60s -i1s

Why is this so? Thx

Are these two iPerf3 runs? And was the first one ~710 and the second around 550?

On the first rund software interrupt processing (SIRQ) shows up as moderate load on core #0 and #1, on te second run it seems appear exclusively on #0 but at much higher magnitude (getting this core close to maxing out). collectd by default samples every 60 seconds, so running your tests for 120 seconds or longer should get easier to interpret results in the graphs.

Thanks for taking a look.

yes that's correct.

I've repeated the tests but run them both for 240seconds and had similar results, the first test reported 709Mbits/sec and the second 563Mbits/sec.

I've attached some more snapshots of graphs and the iperf3 results.


Odd, my hypothesis was that if the SIRQ load maxes out one CPU you would see the slower throughput, but that does not match your second example, even though it is clear that the SIRQ load on #0 is very high in the first test.

Yes very strange.

I'm starting to think this is Windows issue causing the variance but no idea why? I've run the same tests after booting Linux and don't appear to get the two variances like I see on Windows or least I haven't seen it yet. I tried an old W7 machine and it does the same thing as W10.

Hmmm... good that you're trying different machines on the other end.

Check the usage on the Windows machines too. Ive had issues with a 4 core i5 laptop running out of CPU, doing a speedtest on a 1Gb connection. Wasnt expecting that, and am still figuring it out.

In earlier days wasnt able to get full speed on a 300Mb link, turned out that PC couldn't make that on its 1Gb eth card for some reason.

Sometimes theres hidden bottlenecks confounding tests.

It's not the CPU on the Windows machine....it's running at approx. 6% when I do the test, it's a 4 core i7 (3.10Ghz) machine.

This is interesting though.....I did the same test from Ubuntu running in WSL and got better results and no inconsistencies such as those exhibited on W10.CPU on machine at about 10% when using Ubuntu.

One thing you can try to help find the source of the issue is to use udp (-u) , and either set a target bit rate (-b 950M) or as fast is it can go (-b 0). Then run the same test but in reverse (-R).

iperf3.exe -c -t 60 -i 1 -u -b 0 -R

I've found that some devices have transmit bottlenecks for whatever reason, and that slows down the response packets on the tcp test, which can reduce the bandwidth. If iperf3 is running on the router itself you will see drastic differences depending on which direction you test. The router cpu will max out trying to generate packets and send them.

Thanks for the suggestion. I ran the tests as follows:

First straight from my PC to the iPerf server running on OpenWrt (iperf3.exe -c -t 130s -i1s -u -b 0) and the bandwidth result was 948Mbits/sec. I ran the tests for 130s so you could see the info on the cpu info on the graphs.

I then ran the test again but with -R (iperf3.exe -c -t 130s -i1s -u -b 0 -R) and the bandwidth result was 348Mbits/sec. These results seem fairly consistent when repeated.

The CPU on OpenWrt doesn't appear to be maxing out and seems approx 55% for both tests (see graph). One thing I did notice though in the results is that the test WITHOUT -R was showing a 60-70% packet loss whereas the test WITH -R showed not packet loss. This was also consistent.


Side note, instead of making screenshots of terminal windows, just copy and paste the content of the terminal window as text, see the following tip:

Please use the "Preformatted text </>" button for logs, scripts, configs and general console output.
Please edit your post accordingly. Thank you! :slight_smile:

Alteratively, just sandwich your quoted text between lines containing 3 backtick characters:
Your text, line 1
line 2
more lines...

which will render as fixed-fontwidth text:

Your text, line 1
line 2
1 more line...

And that allows easy copy and pasting by others.... and it probably is more convenient than having to make screenshots, editing them and then posting.

Will do next time :+1:. I've closed that window now so can't edit the post unfortunately

1 Like

So clearly there's some kind of bottleneck on the router. The udp test showed that the router could only send/recieve ~330Mbps. (which is 35% of the 948Mbps sent)

I would first try to eliminate the unmanaged switch from the equation and see what you get plugged directly into the port.

Regardless of setup though, first make sure you have no VLAN tagged egress from that router port. In other words, make sure all VLANS are NOT a port member, except your untagged VLAN 10 (or whatever ID you assigned for that LAN subnet).
You never really know how an unmanaged switch will handle tagged traffic...

Previously when I realised that the problem was Windows based I sort of settled with the fact that I would just use Linux for all my benchmark testing. Consequently I have actually changed my network a little and my PC is actually plugged into a managed switch with a trunk port back to OpenWrt. So the OpenWrt router port is actually tagging all VLANs back to my managed switch.

Notwithstanding any of those topology changes the fact is that Windows regardless of topology doesn't give consistent performance results (for TCP at least).......either reporting approx. 720Mbits or approx. 550Mbits

I will plug my PC directly into port2 which is VLAN20 and not tagged with any other VLANs and repeat the test.

Just an update.....I ran 4 new tests with my PC plugged directly into port 2 ( of the OpenWrt router. This port is untagged for VLAN20 so no trunking or other switches involved. It basically produced exactly the same results as before. TCP test will run with result of 728Mbits/sec and then 30 seconds later run with 570Mbits/sec and so on. The UDP test was also as before and I've posted results here.

I've removed some of the test lines for brevity.

c:\iPerf3>iperf3.exe -c -t130s -i1s
Connecting to host, port 5201
[  4] local port 63866 connected to port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  86.6 MBytes   726 Mbits/sec
[  4]   1.00-2.00   sec  87.0 MBytes   731 Mbits/sec
[  4]   2.00-3.00   sec  87.2 MBytes   731 Mbits/sec
[  4] 127.00-128.00 sec  87.5 MBytes   734 Mbits/sec
[  4] 128.00-129.00 sec  87.8 MBytes   736 Mbits/sec
[  4] 129.00-130.00 sec  86.1 MBytes   723 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-130.00 sec  11.0 GBytes   728 Mbits/sec                  sender
[  4]   0.00-130.00 sec  11.0 GBytes   728 Mbits/sec                  receiver

iperf Done.

c:\iPerf3>iperf3.exe -c -t130s -i1s
Connecting to host, port 5201
[  4] local port 64037 connected to port 5201
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-1.00   sec  65.5 MBytes   548 Mbits/sec
[  4]   1.00-2.00   sec  65.8 MBytes   552 Mbits/sec
[  4]   2.00-3.00   sec  65.0 MBytes   546 Mbits/sec
[  4] 127.00-128.00 sec  65.6 MBytes   550 Mbits/sec
[  4] 128.00-129.00 sec  65.5 MBytes   551 Mbits/sec
[  4] 129.00-130.00 sec  65.4 MBytes   548 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth
[  4]   0.00-130.00 sec  8.63 GBytes   570 Mbits/sec                  sender
[  4]   0.00-130.00 sec  8.63 GBytes   570 Mbits/sec                  receiver

iperf Done.

c:\iPerf3>iperf3.exe -c -t 130 -i 1 -u -b 0
Connecting to host, port 5201
[  4] local port 56249 connected to port 5201
[ ID] Interval           Transfer     Bandwidth       Total Datagrams
[  4]   0.00-1.00   sec   113 MBytes   950 Mbits/sec  14500
[  4]   1.00-2.00   sec   113 MBytes   948 Mbits/sec  14460
[  4]   2.00-3.00   sec   113 MBytes   948 Mbits/sec  14470
[  4] 127.00-128.00 sec   113 MBytes   948 Mbits/sec  14470
[  4] 128.00-129.00 sec   113 MBytes   948 Mbits/sec  14470
[  4] 129.00-130.00 sec   113 MBytes   948 Mbits/sec  14470
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-130.00 sec  14.3 GBytes   947 Mbits/sec  0.119 ms  1266833/1877737 (67%)
[  4] Sent 1877737 datagrams

iperf Done.

c:\iPerf3>iperf3.exe -c -t 130 -i 1 -u -b 0 -R
Connecting to host, port 5201
Reverse mode, remote host is sending
[  4] local port 50390 connected to port 5201
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-1.00   sec  47.1 MBytes   395 Mbits/sec  0.068 ms  0/6033 (0%)
[  4]   1.00-2.00   sec  47.1 MBytes   395 Mbits/sec  0.083 ms  0/6031 (0%)
[  4]   2.00-3.00   sec  47.0 MBytes   394 Mbits/sec  0.062 ms  0/6014 (0%)
[  4] 127.00-128.00 sec  46.1 MBytes   387 Mbits/sec  0.083 ms  0/5904 (0%)
[  4] 128.00-129.00 sec  46.2 MBytes   388 Mbits/sec  0.061 ms  0/5918 (0%)
[  4] 129.00-130.00 sec  46.3 MBytes   389 Mbits/sec  0.069 ms  0/5929 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Jitter    Lost/Total Datagrams
[  4]   0.00-130.00 sec  5.30 GBytes   350 Mbits/sec  0.068 ms  3/695000 (0.00043%)
[  4] Sent 695000 datagrams

iperf Done.