for fun, could you also grab an uptime while doing one of those iperfs (an iperf to your isp) just to see the load average? (Perhaps one of the experts can suggest a more informative text based output show cpu usage).
Just tell me how?
Sorry.... Linux Noob.....
I make every Bench you want and post the Date.
Cou helped me so much and really fast and friendly.
simple way is two ssh sessions. On one iperf to your isp. On the other ssh session, do
uptime
near the end of the iperf run
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.03 GBytes 17.4 Gbits/sec 3 1.95 MBytes
[ 5] 1.00-2.00 sec 2.51 GBytes 21.5 Gbits/sec 364 1.13 MBytes
[ 5] 2.00-3.00 sec 2.48 GBytes 21.3 Gbits/sec 199 1.96 MBytes
[ 5] 3.00-4.00 sec 2.56 GBytes 22.0 Gbits/sec 158 1.44 MBytes
[ 5] 4.00-5.00 sec 2.62 GBytes 22.5 Gbits/sec 245 1.81 MBytes
[ 5] 5.00-6.00 sec 2.71 GBytes 23.3 Gbits/sec 162 2.51 MBytes
[ 5] 6.00-7.00 sec 2.74 GBytes 23.5 Gbits/sec 0 3.00 MBytes
[ 5] 7.00-8.00 sec 2.39 GBytes 20.5 Gbits/sec 879 1.84 MBytes
[ 5] 8.00-9.00 sec 2.70 GBytes 23.2 Gbits/sec 33 2.31 MBytes
[ 5] 9.00-10.00 sec 2.46 GBytes 21.1 Gbits/sec 726 1.20 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 25.2 GBytes 21.6 Gbits/sec 2769 sender
[ 5] 0.00-10.05 sec 25.2 GBytes 21.5 Gbits/sec receiver
19:42:04 up 31 min, load average: 0.00, 0.01, 0.00
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.41 GBytes 20.7 Gbits/sec 351 1.76 MBytes
[ 5] 1.00-2.00 sec 2.66 GBytes 22.9 Gbits/sec 677 1.22 MBytes
[ 5] 2.00-3.00 sec 2.56 GBytes 22.0 Gbits/sec 270 1.83 MBytes
[ 5] 3.00-4.00 sec 2.40 GBytes 20.6 Gbits/sec 106 2.09 MBytes
[ 5] 4.00-5.00 sec 2.42 GBytes 20.8 Gbits/sec 171 1.62 MBytes
[ 5] 5.00-6.00 sec 2.64 GBytes 22.7 Gbits/sec 180 1.97 MBytes
[ 5] 6.00-7.00 sec 2.73 GBytes 23.4 Gbits/sec 0 2.50 MBytes
[ 5] 7.00-8.00 sec 2.74 GBytes 23.5 Gbits/sec 0 2.93 MBytes
[ 5] 8.00-9.00 sec 2.52 GBytes 21.7 Gbits/sec 592 1.25 MBytes
[ 5] 9.00-10.00 sec 1.85 GBytes 15.9 Gbits/sec 402 1.22 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 24.9 GBytes 21.4 Gbits/sec 2749 sender
[ 5] 0.00-10.04 sec 24.9 GBytes 21.3 Gbits/sec receiver
19:44:17 up 33 min, load average: 0.08, 0.03, 0.00
1 Like
hmm, that seems a little too good to be true. My suggested observation method must be bad. Perhaps one of the other posters can suggest a better way to see cpu usage while testing throughput.
Thank you for trying it.
i see. But when i run i perf on the box it doesnt go though Openwrt?
Because when i try a Speedtest from a local machine i get very different results... (Info, this machine is connected to the box with 10gbps Fiber not 25gbps... )
"on the box" i assume is the box running openwrt. You can run iperf from openwrt or from a client. I am curious about the cpu usage on the device running openwrt when you iperf to your isp through openwrt. (looking at your log results perhaps try running iperf longer via iperf -t 300 <isp iperf server>)
a better way might be (from a ssh session to the openwrt device)
opkg update
opkg install mpstat
and then run your iperf.
near the end of the perf (but before it stops) run on a dedicated ssh session to the openwrt device
mpstat -A
if your cpu usage is that good, then I think that's pretty cool.
Thanks again for testing.
I have two ssh sessions ( Putty ) open to my openwrt device.
And tested it from there.
mpstat returned an error that the package is unknown
my bad, it looks like /bin/mpstat on my openwrt device is provided by busy box. Not sure if I added that in or if openwrt provides it by default. Thanks again for testing
Irqballance split irq between threads , most nice network cards have more than 1tx 1 rx queue (irq) , like intel x520 10gb card have 8rx and 8tx irq this helos a lot to soki load and make benefit from multithread cpus. Don't remember how much have i710 npu but tgose are nicely optimised . will check my 40gbe cards later just out of curioysity.
P.s real performanxe You can see while torrenting on private sites.
1 Like
What an amazing idea, efahl! With the iperf.. For the benefit of the community, here are some stats of various hardware that I had around. All tests ran with the -P number_of_threads that the CPU had (iperf3 -c 127.0.0.1 -P #).
Raspberry PI 4b (-P 4)
Raspbian Linux: 1.3 to 1.8 Gbits / sec
Beelink U59 (N5105, 2 Realtek Nics 1000Mbps) (-P 4)
OPNsense: 4.15 Gbits/sec
IPFire linux: 7.37 Gbits/sec
J4125 CPU (-P 4)
OPNsense: 2 Gbits/sec
IPfire linux: 5.5 Gbits/sec
Intel 8700 CPU (-P 12)
Linux: 4.9 Gbits/sec
Is this real iperf3 from OpenWrt box to external ISP iperf3 server?
If yes, that it. You can almost saturate 25Gbit. Which is amazing.
It would be nice if you can run two SSH sessions, one running iperf3 at full blast and another running htop so we can see CPU core times.
Yes and no: yes, it's real iperf3 on the OpenWrt box, but no, it's a loopback test with both instances of iperf3 on that OpenWrt box. This bypasses the NICs and their drivers, and gives a rough indication about the performance of your router's network stack. As with any synthetic benchmark, huge caveats apply, but it is useful as a gross gauge as to potential (say, if your results show a given device can only touch 900 Mbps internally, there is no way it's going to give you 1 Gbps on your shiny new FTTH service).
Here's a whole thread dedicated to running the loopback test on various hardware:
As they also touched in that thread, running iperf3 on loopback interface is quite pointless and will often yield 10x than real life server-to-server. It is basically a CPU cache test.
2 Likes
@cryoplay Which was in the end the right package that you installed for your " Intel XXV710 Dual SFP28 NIC" card? As I am looking right now into buying my hardware for my new 25gig internet uplink and it needs to be OpenWRT compatible 
thanks!