OpenWrt for 25Gbit Internet possible?

for fun, could you also grab an uptime while doing one of those iperfs (an iperf to your isp) just to see the load average? (Perhaps one of the experts can suggest a more informative text based output show cpu usage).

Just tell me how?

Sorry.... Linux Noob.....

I make every Bench you want and post the Date.
Cou helped me so much and really fast and friendly.

simple way is two ssh sessions. On one iperf to your isp. On the other ssh session, do

uptime

near the end of the iperf run

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.03 GBytes  17.4 Gbits/sec    3   1.95 MBytes
[  5]   1.00-2.00   sec  2.51 GBytes  21.5 Gbits/sec  364   1.13 MBytes
[  5]   2.00-3.00   sec  2.48 GBytes  21.3 Gbits/sec  199   1.96 MBytes
[  5]   3.00-4.00   sec  2.56 GBytes  22.0 Gbits/sec  158   1.44 MBytes
[  5]   4.00-5.00   sec  2.62 GBytes  22.5 Gbits/sec  245   1.81 MBytes
[  5]   5.00-6.00   sec  2.71 GBytes  23.3 Gbits/sec  162   2.51 MBytes
[  5]   6.00-7.00   sec  2.74 GBytes  23.5 Gbits/sec    0   3.00 MBytes
[  5]   7.00-8.00   sec  2.39 GBytes  20.5 Gbits/sec  879   1.84 MBytes
[  5]   8.00-9.00   sec  2.70 GBytes  23.2 Gbits/sec   33   2.31 MBytes
[  5]   9.00-10.00  sec  2.46 GBytes  21.1 Gbits/sec  726   1.20 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  25.2 GBytes  21.6 Gbits/sec  2769             sender
[  5]   0.00-10.05  sec  25.2 GBytes  21.5 Gbits/sec                  receiver

19:42:04 up 31 min, load average: 0.00, 0.01, 0.00

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  2.41 GBytes  20.7 Gbits/sec  351   1.76 MBytes
[  5]   1.00-2.00   sec  2.66 GBytes  22.9 Gbits/sec  677   1.22 MBytes
[  5]   2.00-3.00   sec  2.56 GBytes  22.0 Gbits/sec  270   1.83 MBytes
[  5]   3.00-4.00   sec  2.40 GBytes  20.6 Gbits/sec  106   2.09 MBytes
[  5]   4.00-5.00   sec  2.42 GBytes  20.8 Gbits/sec  171   1.62 MBytes
[  5]   5.00-6.00   sec  2.64 GBytes  22.7 Gbits/sec  180   1.97 MBytes
[  5]   6.00-7.00   sec  2.73 GBytes  23.4 Gbits/sec    0   2.50 MBytes
[  5]   7.00-8.00   sec  2.74 GBytes  23.5 Gbits/sec    0   2.93 MBytes
[  5]   8.00-9.00   sec  2.52 GBytes  21.7 Gbits/sec  592   1.25 MBytes
[  5]   9.00-10.00  sec  1.85 GBytes  15.9 Gbits/sec  402   1.22 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  24.9 GBytes  21.4 Gbits/sec  2749             sender
[  5]   0.00-10.04  sec  24.9 GBytes  21.3 Gbits/sec                  receiver

19:44:17 up 33 min,  load average: 0.08, 0.03, 0.00
1 Like

hmm, that seems a little too good to be true. My suggested observation method must be bad. Perhaps one of the other posters can suggest a better way to see cpu usage while testing throughput.

Thank you for trying it.

i see. But when i run i perf on the box it doesnt go though Openwrt?

Because when i try a Speedtest from a local machine i get very different results... (Info, this machine is connected to the box with 10gbps Fiber not 25gbps... )

Made the same iperf test and looked at the logs

"on the box" i assume is the box running openwrt. You can run iperf from openwrt or from a client. I am curious about the cpu usage on the device running openwrt when you iperf to your isp through openwrt. (looking at your log results perhaps try running iperf longer via iperf -t 300 <isp iperf server>)

a better way might be (from a ssh session to the openwrt device)

opkg update
opkg install mpstat

and then run your iperf.
near the end of the perf (but before it stops) run on a dedicated ssh session to the openwrt device

mpstat -A

if your cpu usage is that good, then I think that's pretty cool.

Thanks again for testing.

I have two ssh sessions ( Putty ) open to my openwrt device.
And tested it from there.

mpstat returned an error that the package is unknown

my bad, it looks like /bin/mpstat on my openwrt device is provided by busy box. Not sure if I added that in or if openwrt provides it by default. Thanks again for testing

Irqballance split irq between threads , most nice network cards have more than 1tx 1 rx queue (irq) , like intel x520 10gb card have 8rx and 8tx irq this helos a lot to soki load and make benefit from multithread cpus. Don't remember how much have i710 npu but tgose are nicely optimised . will check my 40gbe cards later just out of curioysity.

P.s real performanxe You can see while torrenting on private sites.

1 Like

What an amazing idea, efahl! With the iperf.. For the benefit of the community, here are some stats of various hardware that I had around. All tests ran with the -P number_of_threads that the CPU had (iperf3 -c 127.0.0.1 -P #).

Raspberry PI 4b (-P 4)

Raspbian Linux: 1.3 to 1.8 Gbits / sec

Beelink U59 (N5105, 2 Realtek Nics 1000Mbps) (-P 4)

OPNsense: 4.15 Gbits/sec
IPFire linux: 7.37 Gbits/sec

J4125 CPU (-P 4)

OPNsense: 2 Gbits/sec
IPfire linux: 5.5 Gbits/sec

Intel 8700 CPU (-P 12)

Linux: 4.9 Gbits/sec

Is this real iperf3 from OpenWrt box to external ISP iperf3 server?
If yes, that it. You can almost saturate 25Gbit. Which is amazing.

It would be nice if you can run two SSH sessions, one running iperf3 at full blast and another running htop so we can see CPU core times.

Yes and no: yes, it's real iperf3 on the OpenWrt box, but no, it's a loopback test with both instances of iperf3 on that OpenWrt box. This bypasses the NICs and their drivers, and gives a rough indication about the performance of your router's network stack. As with any synthetic benchmark, huge caveats apply, but it is useful as a gross gauge as to potential (say, if your results show a given device can only touch 900 Mbps internally, there is no way it's going to give you 1 Gbps on your shiny new FTTH service).

Here's a whole thread dedicated to running the loopback test on various hardware:

As they also touched in that thread, running iperf3 on loopback interface is quite pointless and will often yield 10x than real life server-to-server. It is basically a CPU cache test.

2 Likes