M-hmm. I see.
which version of Openwrt are you running on both?
I have a Beelink EQ12 with 16GB DDR5 and the Alder Lake N100 processor running proxmox 8.
Here is OpenWRT 23.05.3 running in an LXC assigned 2 (of 4) cores:
root@openwrt:/# uname -a
Linux openwrt 6.8.4-3-pve #1 SMP PREEMPT_DYNAMIC PMX 6.8.4-3 (2024-05-02T11:55Z) x86_64 GNU/Linux
root@openwrt:/# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 45364 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.00 GBytes 25.7 Gbits/sec 0 14.7 MBytes
[ 5] 1.00-2.00 sec 2.88 GBytes 24.8 Gbits/sec 0 16.2 MBytes
[ 5] 2.00-3.00 sec 2.98 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 3.00-4.00 sec 2.99 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 4.00-5.00 sec 2.98 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 5.00-6.00 sec 2.96 GBytes 25.4 Gbits/sec 0 16.2 MBytes
[ 5] 6.00-7.00 sec 2.99 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 7.00-8.00 sec 2.98 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 8.00-9.00 sec 2.98 GBytes 25.6 Gbits/sec 0 16.2 MBytes
[ 5] 9.00-10.00 sec 2.98 GBytes 25.6 Gbits/sec 0 16.2 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 29.7 GBytes 25.5 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 29.7 GBytes 25.5 Gbits/sec receiver
iperf Done.
And then run on the native proxmox with no restrictions:
root@pve07:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 36448 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 5.17 GBytes 44.4 Gbits/sec 0 3.43 MBytes
[ 5] 1.00-2.00 sec 5.40 GBytes 46.4 Gbits/sec 0 3.43 MBytes
[ 5] 2.00-3.00 sec 5.59 GBytes 48.0 Gbits/sec 0 3.56 MBytes
[ 5] 3.00-4.00 sec 5.70 GBytes 49.0 Gbits/sec 0 3.56 MBytes
[ 5] 4.00-5.00 sec 5.33 GBytes 45.8 Gbits/sec 0 3.56 MBytes
[ 5] 5.00-6.00 sec 5.36 GBytes 46.0 Gbits/sec 0 3.56 MBytes
[ 5] 6.00-7.00 sec 5.48 GBytes 47.1 Gbits/sec 0 3.56 MBytes
[ 5] 7.00-8.00 sec 5.45 GBytes 46.8 Gbits/sec 0 3.56 MBytes
[ 5] 8.00-9.00 sec 5.67 GBytes 48.7 Gbits/sec 0 3.56 MBytes
[ 5] 9.00-10.00 sec 5.68 GBytes 48.8 Gbits/sec 0 3.56 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 54.8 GBytes 47.1 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 54.8 GBytes 47.1 Gbits/sec receiver
iperf Done.
I would expect that LXC has some overhead but this seems like more than I would hope. The bare metal run is almost 2X the performance of the LXC. Maybe it is just the # of cores assigned?
Even for the danger of sounding like a broken record, running OpenWrt as a container is not a supported configuration and tears up huge security issues and other bugs (ranging from subtle to glaringly obvious ones). The forum search will give you the reasons why, but really - never do that.
Running OpenWrt in full system virtualization (qemu-kvm, virtualbox, hyper-v, vmware, etc.) or emulation (plain qemu) is fine, running it as a container is not.
I haven't seen anything convincing as to reasons to not run OpenWRT on LXC and I've done a ton of searching. Mostly I've seen weak arguments.
EDIT: OK OK, I will be a compliant el-user and use a VM. I imported the official EFI VM into proxmox using this guide backed up my LXC config and restored it to the VM.
With the same 2CPU assignment, 8GB ext4 disk, 512MB RAM: I get the following iperf internel results on the VM:
-----------------------------------
OpenWrt 23.05.3, r23809-234f1a2efa
-----------------------------------
root@OpenWrt:/# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 44446 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 7.04 GBytes 60.3 Gbits/sec 0 1.62 MBytes
[ 5] 1.00-2.00 sec 6.73 GBytes 57.8 Gbits/sec 0 1.94 MBytes
[ 5] 2.00-3.00 sec 7.25 GBytes 62.3 Gbits/sec 0 1.94 MBytes
[ 5] 3.00-4.00 sec 7.25 GBytes 62.3 Gbits/sec 0 1.94 MBytes
[ 5] 4.00-5.00 sec 7.24 GBytes 62.3 Gbits/sec 0 1.94 MBytes
[ 5] 5.00-6.00 sec 7.14 GBytes 61.4 Gbits/sec 0 1.94 MBytes
[ 5] 6.00-7.00 sec 6.45 GBytes 55.3 Gbits/sec 0 1.94 MBytes
[ 5] 7.00-8.00 sec 6.62 GBytes 56.9 Gbits/sec 0 1.94 MBytes
[ 5] 8.00-9.00 sec 7.24 GBytes 62.2 Gbits/sec 0 1.94 MBytes
[ 5] 9.00-10.00 sec 7.26 GBytes 62.4 Gbits/sec 0 1.94 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 70.2 GBytes 60.3 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 70.2 GBytes 60.2 Gbits/sec receiver
iperf Done.
Initial tests using bridged LAN and WAN it seems to perform similarly in real-world applications as well as running speed tests on WAN and LAN. I am perhaps losing a little bit of performance on the 2.5Gb link between my desktop and the EQ12 but it is still averaging 2.37 Gbs
So, basically the internel iperf3 loopback performance seems kinda random and perhaps not very relevant to real-world performance. It's hard for me to tell what sort of overall CPU load this is putting on the proxmox node, but seems pretty light watching the CPU load graph.
I was able to get proxmox to enable the IOMMU and load the virtio modules, so it should be possible to pass the WAN device into the VM and be #securesmaxxing
EDIT2: with only a little finagling, I got the PCIe passthrough working to put the 2nd Intel i225-v b3 NIC as PCI. This makes the NIC invisible to the proxmox host and other VMs / LXCs which is the maximum security available for a virtualized gateway router. Overall I am pretty happy with this little Beelink EQ12. TCP offloading works fine so far, will see if stability problems pop up over time.
1 Like
root@OpenWrt:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 38822 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 962 MBytes 8.06 Gbits/sec 0 1.25 MBytes
[ 5] 1.00-2.00 sec 1010 MBytes 8.48 Gbits/sec 0 1.25 MBytes
[ 5] 2.00-3.00 sec 958 MBytes 8.04 Gbits/sec 0 1.25 MBytes
[ 5] 3.00-4.00 sec 1006 MBytes 8.44 Gbits/sec 0 1.25 MBytes
[ 5] 4.00-5.00 sec 968 MBytes 8.12 Gbits/sec 0 1.25 MBytes
[ 5] 5.00-6.00 sec 998 MBytes 8.37 Gbits/sec 0 1.25 MBytes
[ 5] 6.00-7.00 sec 924 MBytes 7.75 Gbits/sec 0 1.25 MBytes
[ 5] 7.00-8.00 sec 979 MBytes 8.21 Gbits/sec 0 1.25 MBytes
[ 5] 8.00-9.00 sec 974 MBytes 8.17 Gbits/sec 0 1.25 MBytes
[ 5] 9.00-10.00 sec 988 MBytes 8.29 Gbits/sec 0 1.25 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 9.86 GBytes 8.47 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 9.86 GBytes 8.47 Gbits/sec receiver
iperf Done.
root@OpenWrt:~# ubus call system board
{
"kernel": "5.15.162",
"hostname": "OpenWrt",
"system": "Intel(R) Atom(TM) x5-Z8300 CPU @ 1.44GHz",
"model": "CSL Computer Narrow Box 4K",
"board_name": "csl-computer-narrow-box-4k",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05.4",
"revision": "r24012-d8dd03c46f",
"target": "x86/generic",
"description": "OpenWrt 23.05.4 r24012-d8dd03c46f"
}
}
The test suggested by this thread turns out to be a very good test case for CPU stability. Issues can't find by 'openssl speed' or 'stress' will be revealed by this local loop. Any variation in the speed seems indicative of stability issue. This conclusion, of coz, does not apply to x86 CPUs which come with crazy boost algorithm.
EDIT: Spoke too soon. seems some 'stability issue' in the network stack but not necessarily hardware e.g. CPU clock or stability per se.
Not quite, the base CPU load isn't static - there are always cyclic and random tasks the kernel/ CPU has to take care of, which takes away some processing power for other needs. You will see some variation by definition.
Sorry for the belated response. I wanted to sort out issues I saw in the test first before I speak again.
I agree with what you said. However, what you said is only a trivial true case. Most likely irrelevant for what users will see in real tests. Perhaps I should have quantified what I meant by "Any variation in the speed".
"Any variation" was a strong phrase and I should have avoided using it that may lead to misinterpretation. I think with less than 5% should consider fine, covering the trivial case you described. Variability higher than 5% flags potential issues in the system, configuration (or any hardware overlock).
I saw higher than 5% variation the other day. I was pretty sure it didn't fall into hopping cores, kernel tasks or whatever. My BPI-R4 was pretty much idle with <1% CPU utilisation.
I'm going to propose a modified test that's proper and better, avoiding the exact variation that I saw in the next post.
root@OpenWrt:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 37046 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 4.73 GBytes 40.5 Gbits/sec 0 639 KBytes
[ 5] 1.00-2.00 sec 4.77 GBytes 40.9 Gbits/sec 0 1.44 MBytes
[ 5] 2.00-3.00 sec 4.83 GBytes 41.5 Gbits/sec 0 2.00 MBytes
[ 5] 3.00-4.00 sec 3.78 GBytes 32.5 Gbits/sec 0 2.00 MBytes
[ 5] 4.00-5.00 sec 2.85 GBytes 24.5 Gbits/sec 0 2.00 MBytes
[ 5] 5.00-6.00 sec 1.08 GBytes 9.25 Gbits/sec 0 2.00 MBytes
[ 5] 6.00-7.00 sec 1.92 GBytes 16.5 Gbits/sec 0 2.00 MBytes
[ 5] 7.00-8.00 sec 2.64 GBytes 22.6 Gbits/sec 0 2.00 MBytes
[ 5] 8.00-9.00 sec 5.15 GBytes 44.2 Gbits/sec 0 2.00 MBytes
[ 5] 9.00-10.00 sec 4.36 GBytes 37.5 Gbits/sec 0 2.00 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 50.6 GBytes 43.4 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 50.6 GBytes 43.4 Gbits/sec receiver
iperf Done.
root@OpenWrt:~# ubus call system board
{
"kernel": "5.15.163",
"hostname": "OpenWrt",
"system": "AMD Ryzen 7 5700U with Radeon Graphics",
"model": "HC Technology.,Ltd. HCAR5000-MI2",
"board_name": "hc-technology-ltd-hcar5000-mi2",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05-SNAPSHOT",
"revision": "r24016-f791ec1f6d",
"target": "x86/64",
"description": "OpenWrt 23.05-SNAPSHOT r24016-f791ec1f6d"
}
}
root@GL-MT6000:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 41644 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 927 MBytes 7.77 Gbits/sec 0 1023 KBytes
[ 5] 1.00-2.00 sec 780 MBytes 6.54 Gbits/sec 0 1023 KBytes
[ 5] 2.00-3.00 sec 926 MBytes 7.77 Gbits/sec 0 1023 KBytes
[ 5] 3.00-4.00 sec 924 MBytes 7.75 Gbits/sec 0 1023 KBytes
[ 5] 4.00-5.00 sec 931 MBytes 7.81 Gbits/sec 0 1023 KBytes
[ 5] 5.00-6.00 sec 830 MBytes 6.97 Gbits/sec 0 1023 KBytes
[ 5] 6.00-7.00 sec 755 MBytes 6.34 Gbits/sec 0 1023 KBytes
[ 5] 7.00-8.00 sec 752 MBytes 6.31 Gbits/sec 0 1023 KBytes
[ 5] 8.00-9.00 sec 752 MBytes 6.31 Gbits/sec 0 1023 KBytes
[ 5] 9.00-10.00 sec 754 MBytes 6.32 Gbits/sec 0 1023 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 8.14 GBytes 6.99 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 8.13 GBytes 6.98 Gbits/sec receiver
iperf Done.
root@GL-MT6000:~# ubus call system board
{
"kernel": "6.6.32",
"hostname": "GL-MT6000",
"system": "ARMv8 Processor rev 4",
"model": "GL.iNet GL-MT6000",
"board_name": "glinet,gl-mt6000",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "24.0",
"revision": "r26550+32-eea666d583",
"target": "mediatek/filogic",
"description": "OpenWrt 24.0 r26550+32-eea666d583"
}
}
root@BananaWRT:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 50416 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.19 GBytes 18.8 Gbits/sec 0 895 KBytes
[ 5] 1.00-2.00 sec 2.24 GBytes 19.2 Gbits/sec 0 1.19 MBytes
[ 5] 2.00-3.00 sec 2.24 GBytes 19.3 Gbits/sec 0 1.19 MBytes
[ 5] 3.00-4.00 sec 2.24 GBytes 19.3 Gbits/sec 0 1.19 MBytes
[ 5] 4.00-5.00 sec 2.26 GBytes 19.4 Gbits/sec 0 1.19 MBytes
[ 5] 5.00-6.00 sec 2.24 GBytes 19.3 Gbits/sec 0 1.19 MBytes
[ 5] 6.00-7.00 sec 2.24 GBytes 19.2 Gbits/sec 0 1.19 MBytes
[ 5] 7.00-8.00 sec 2.24 GBytes 19.3 Gbits/sec 0 1.19 MBytes
[ 5] 8.00-9.00 sec 2.25 GBytes 19.3 Gbits/sec 0 1.19 MBytes
[ 5] 9.00-10.00 sec 2.24 GBytes 19.3 Gbits/sec 0 1.19 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 22.4 GBytes 19.2 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 22.4 GBytes 19.2 Gbits/sec receiver
iperf Done.
root@BananaWRT:~# ubus call system board
{
"kernel": "6.6.41",
"hostname": "BananaWRT",
"system": "ARMv8 Processor rev 0",
"model": "Bananapi BPI-R4",
"board_name": "bananapi,bpi-r4",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "SNAPSHOT",
"revision": "r26997-a67b20e3d5",
"target": "mediatek/filogic",
"description": "OpenWrt SNAPSHOT r26997-a67b20e3d5"
}
}
root@BananaWRT:~#
-------------------------------------------------------------------------------------------
root@BananaWRT:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 41276 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 2.44 GBytes 20.9 Gbits/sec 0 959 KBytes
[ 5] 1.00-2.00 sec 2.46 GBytes 21.1 Gbits/sec 0 959 KBytes
[ 5] 2.00-3.00 sec 2.49 GBytes 21.4 Gbits/sec 0 959 KBytes
[ 5] 3.00-4.00 sec 2.46 GBytes 21.1 Gbits/sec 0 959 KBytes
[ 5] 4.00-5.00 sec 2.46 GBytes 21.2 Gbits/sec 0 959 KBytes
[ 5] 5.00-6.00 sec 2.36 GBytes 20.3 Gbits/sec 0 959 KBytes
[ 5] 6.00-7.00 sec 2.51 GBytes 21.6 Gbits/sec 0 959 KBytes
[ 5] 7.00-8.00 sec 2.50 GBytes 21.5 Gbits/sec 0 959 KBytes
[ 5] 8.00-9.00 sec 2.54 GBytes 21.8 Gbits/sec 0 959 KBytes
[ 5] 9.00-10.00 sec 2.43 GBytes 20.8 Gbits/sec 0 959 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 25.1 GBytes 21.5 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 25.1 GBytes 21.5 Gbits/sec receiver
iperf Done.
root@BananaWRT:~# ubus call system board
{
"kernel": "6.6.43",
"hostname": "BananaWRT",
"system": "ARMv8 Processor rev 0",
"model": "Bananapi BPI-R4",
"board_name": "bananapi,bpi-r4",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "SNAPSHOT",
"revision": "r27064-f20987c161",
"target": "mediatek/filogic",
"description": "OpenWrt SNAPSHOT r27064-f20987c161"
}
}
root@BananaWRT:~#
Try this alternative test with UDP (instead of TCP):
root@beepi:~$ iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killall iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 36495 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 4.02 GBytes 34.5 Gbits/sec 0.002 ms 1/131833 (0.00076%)
[ 5] 1.00-2.00 sec 4.04 GBytes 34.7 Gbits/sec 0.002 ms 0/132305 (0%)
[ 5] 2.00-3.00 sec 4.03 GBytes 34.6 Gbits/sec 0.003 ms 0/132173 (0%)
[ 5] 3.00-4.00 sec 4.02 GBytes 34.6 Gbits/sec 0.002 ms 0/131826 (0%)
[ 5] 4.00-5.00 sec 4.03 GBytes 34.6 Gbits/sec 0.003 ms 0/131973 (0%)
[ 5] 5.00-6.00 sec 4.05 GBytes 34.8 Gbits/sec 0.002 ms 0/132841 (0%)
[ 5] 6.00-7.00 sec 4.02 GBytes 34.5 Gbits/sec 0.002 ms 3/131651 (0.0023%)
[ 5] 7.00-8.00 sec 4.06 GBytes 34.8 Gbits/sec 0.003 ms 105/133110 (0.079%)
[ 5] 8.00-9.00 sec 4.06 GBytes 34.9 Gbits/sec 0.002 ms 0/132967 (0%)
[ 5] 9.00-10.00 sec 4.05 GBytes 34.8 Gbits/sec 0.003 ms 0/132831 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 40.4 GBytes 34.7 Gbits/sec 0.000 ms 0/1323522 (0%) sender
[ 5] 0.00-10.00 sec 40.4 GBytes 34.7 Gbits/sec 0.003 ms 109/1323510 (0.0082%) receiver
iperf Done.
root@beepi:~$ ubus call system board
{
"kernel": "6.6.43",
"hostname": "beepi",
"system": "ARMv8 Processor rev 0",
"model": "Bananapi BPI-R4",
"board_name": "bananapi,bpi-r4",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "SNAPSHOT",
"revision": "r0-ca446904",
"target": "mediatek/filogic",
"description": "OpenWrt SNAPSHOT r0-ca446904"
}
}
1 Like
On a proxmox (kvm) vm with 2 vcpu, type IvyBridge-IBRS, numa enabled on e5-2630v2 x2
iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 44166 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.53 GBytes 30.3 Gbits/sec 0 8.06 MBytes
[ 5] 1.00-2.00 sec 3.51 GBytes 30.1 Gbits/sec 0 8.06 MBytes
[ 5] 2.00-3.00 sec 3.49 GBytes 30.0 Gbits/sec 0 8.06 MBytes
[ 5] 3.00-4.00 sec 3.55 GBytes 30.5 Gbits/sec 0 8.06 MBytes
[ 5] 4.00-5.00 sec 3.57 GBytes 30.7 Gbits/sec 0 8.06 MBytes
[ 5] 5.00-6.00 sec 3.45 GBytes 29.7 Gbits/sec 0 8.06 MBytes
[ 5] 6.00-7.00 sec 3.52 GBytes 30.2 Gbits/sec 0 8.06 MBytes
[ 5] 7.00-8.00 sec 3.49 GBytes 30.0 Gbits/sec 0 8.06 MBytes
[ 5] 8.00-9.00 sec 3.24 GBytes 27.9 Gbits/sec 0 8.06 MBytes
[ 5] 9.00-10.00 sec 3.50 GBytes 30.0 Gbits/sec 0 8.06 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 35.4 GBytes 30.4 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 35.4 GBytes 30.4 Gbits/sec receiver
iperf Done.
iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killall iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 51596 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 2.25 GBytes 19.3 Gbits/sec 0.006 ms 11/73751 (0.015%)
[ 5] 1.00-2.00 sec 2.28 GBytes 19.6 Gbits/sec 0.008 ms 0/74862 (0%)
[ 5] 2.00-3.00 sec 2.29 GBytes 19.6 Gbits/sec 0.006 ms 0/74910 (0%)
[ 5] 3.00-4.00 sec 2.29 GBytes 19.6 Gbits/sec 0.008 ms 11/74951 (0.015%)
[ 5] 4.00-5.00 sec 2.29 GBytes 19.7 Gbits/sec 0.007 ms 8/75036 (0.011%)
[ 5] 5.00-6.00 sec 2.29 GBytes 19.7 Gbits/sec 0.007 ms 56/75252 (0.074%)
[ 5] 6.00-7.00 sec 2.29 GBytes 19.7 Gbits/sec 0.008 ms 12/74984 (0.016%)
[ 5] 7.00-8.00 sec 2.30 GBytes 19.8 Gbits/sec 0.007 ms 0/75393 (0%)
[ 5] 8.00-9.00 sec 2.30 GBytes 19.8 Gbits/sec 0.008 ms 0/75342 (0%)
[ 5] 9.00-10.00 sec 2.30 GBytes 19.7 Gbits/sec 0.006 ms 0/75251 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 22.9 GBytes 19.7 Gbits/sec 0.000 ms 0/749751 (0%) sender
[ 5] 0.00-10.00 sec 22.9 GBytes 19.6 Gbits/sec 0.006 ms 98/749732 (0.013%) receiver
iperf Done.
Routerich AX3000
stock firmware (64mb ubi/overlay storage)
OpenWrt 23.05.3 based
# ubus call system board
{
"kernel": "5.15.150",
"hostname": "RouteRich",
"system": "ARMv8 Processor rev 4",
"model": "Routerich AX3000",
"board_name": "routerich,ax3000",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05.3",
"revision": "r23809-234f1a2efa",
"target": "mediatek/filogic",
"description": "OpenWrt 23.05.3 r23809-234f1a2efa"
}
}
# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 55180 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 570 MBytes 4.78 Gbits/sec 0 1.06 MBytes
[ 5] 1.00-2.00 sec 577 MBytes 4.84 Gbits/sec 0 1.06 MBytes
[ 5] 2.00-3.00 sec 564 MBytes 4.73 Gbits/sec 0 1.06 MBytes
[ 5] 3.00-4.00 sec 584 MBytes 4.90 Gbits/sec 0 1.06 MBytes
[ 5] 4.00-5.00 sec 578 MBytes 4.85 Gbits/sec 0 1.06 MBytes
[ 5] 5.00-6.00 sec 590 MBytes 4.95 Gbits/sec 0 1.06 MBytes
[ 5] 6.00-7.00 sec 568 MBytes 4.76 Gbits/sec 0 1.06 MBytes
[ 5] 7.00-8.00 sec 568 MBytes 4.76 Gbits/sec 0 1.06 MBytes
[ 5] 8.00-9.00 sec 564 MBytes 4.73 Gbits/sec 0 1.06 MBytes
[ 5] 9.00-10.00 sec 554 MBytes 4.65 Gbits/sec 0 1.06 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 5.59 GBytes 4.80 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 5.59 GBytes 4.80 Gbits/sec receiver
iperf Done.
# iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killall iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 56136 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 794 MBytes 6.66 Gbits/sec 0.014 ms 432/25875 (1.7%)
[ 5] 1.00-2.00 sec 797 MBytes 6.69 Gbits/sec 0.017 ms 405/25930 (1.6%)
[ 5] 2.00-3.00 sec 782 MBytes 6.56 Gbits/sec 0.014 ms 853/25911 (3.3%)
[ 5] 3.00-4.00 sec 794 MBytes 6.66 Gbits/sec 0.015 ms 227/25663 (0.88%)
[ 5] 4.00-5.00 sec 792 MBytes 6.65 Gbits/sec 0.012 ms 255/25629 (0.99%)
[ 5] 5.00-6.00 sec 779 MBytes 6.54 Gbits/sec 0.020 ms 1001/25951 (3.9%)
[ 5] 6.00-7.00 sec 791 MBytes 6.63 Gbits/sec 0.015 ms 506/25836 (2%)
[ 5] 7.00-8.00 sec 789 MBytes 6.62 Gbits/sec 0.012 ms 552/25835 (2.1%)
[ 5] 8.00-9.00 sec 791 MBytes 6.63 Gbits/sec 0.014 ms 543/25861 (2.1%)
[ 5] 9.00-10.00 sec 792 MBytes 6.65 Gbits/sec 0.014 ms 432/25806 (1.7%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 7.88 GBytes 6.76 Gbits/sec 0.000 ms 0/258304 (0%) sender
[ 5] 0.00-10.00 sec 7.72 GBytes 6.63 Gbits/sec 0.014 ms 5206/258297 (2%) receiver
iperf Done.
PC/NAS Chatreey Intel N100
16 Gb RAM
Tested on host Proxmox (Debian based)
# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 44594 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 3.78 GBytes 32.4 Gbits/sec 0 1.69 MBytes
[ 5] 1.00-2.00 sec 2.71 GBytes 23.3 Gbits/sec 0 1.75 MBytes
[ 5] 2.00-3.00 sec 2.55 GBytes 21.9 Gbits/sec 0 1.81 MBytes
[ 5] 3.00-4.00 sec 2.26 GBytes 19.4 Gbits/sec 0 2.69 MBytes
[ 5] 4.00-5.00 sec 1.80 GBytes 15.4 Gbits/sec 0 2.69 MBytes
[ 5] 5.00-6.00 sec 2.29 GBytes 19.7 Gbits/sec 0 4.18 MBytes
[ 5] 6.00-7.00 sec 2.52 GBytes 21.6 Gbits/sec 0 4.18 MBytes
[ 5] 7.00-8.00 sec 2.79 GBytes 24.0 Gbits/sec 0 4.18 MBytes
[ 5] 8.00-9.00 sec 2.69 GBytes 23.1 Gbits/sec 0 4.18 MBytes
[ 5] 9.00-10.00 sec 2.74 GBytes 23.6 Gbits/sec 0 4.18 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 26.1 GBytes 22.4 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 26.1 GBytes 22.4 Gbits/sec receiver
iperf Done.
# iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killall iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 53034 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 2.90 GBytes 24.9 Gbits/sec 0.001 ms 144/95213 (0.15%)
[ 5] 1.00-2.00 sec 2.25 GBytes 19.3 Gbits/sec 0.001 ms 1450/75015 (1.9%)
[ 5] 2.00-3.00 sec 2.08 GBytes 17.9 Gbits/sec 0.003 ms 518/68654 (0.75%)
[ 5] 3.00-4.00 sec 2.46 GBytes 21.1 Gbits/sec 0.005 ms 63/80693 (0.078%)
[ 5] 4.00-5.00 sec 2.08 GBytes 17.9 Gbits/sec 0.005 ms 796/69000 (1.2%)
[ 5] 5.00-6.00 sec 2.33 GBytes 20.1 Gbits/sec 0.005 ms 260/76758 (0.34%)
[ 5] 6.00-7.00 sec 2.19 GBytes 18.9 Gbits/sec 0.158 ms 3109/75027 (4.1%)
[ 5] 7.00-8.00 sec 2.07 GBytes 17.7 Gbits/sec 0.005 ms 1617/69317 (2.3%)
[ 5] 8.00-9.00 sec 2.32 GBytes 20.0 Gbits/sec 0.006 ms 94/76249 (0.12%)
[ 5] 9.00-10.00 sec 2.11 GBytes 18.1 Gbits/sec 0.006 ms 442/69601 (0.64%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 23.1 GBytes 19.8 Gbits/sec 0.000 ms 0/755530 (0%) sender
[ 5] 0.00-10.00 sec 22.8 GBytes 19.6 Gbits/sec 0.006 ms 8493/755527 (1.1%) receiver
iperf Done.
Dell OptiPlex 7050, Core i3-7100T, 8GB RAM, OpenWRT 23.05.2
root@OptiPlex-7050:~# iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killa
ll iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 35765 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 4.14 GBytes 35.6 Gbits/sec 0.002 ms 0/135755 (0%)
[ 5] 1.00-2.00 sec 4.04 GBytes 34.7 Gbits/sec 0.002 ms 0/132331 (0%)
[ 5] 2.00-3.00 sec 3.97 GBytes 34.1 Gbits/sec 0.004 ms 0/130214 (0%)
[ 5] 3.00-4.00 sec 4.03 GBytes 34.7 Gbits/sec 0.003 ms 0/132202 (0%)
[ 5] 4.00-5.00 sec 4.04 GBytes 34.7 Gbits/sec 0.003 ms 0/132407 (0%)
[ 5] 5.00-6.00 sec 4.04 GBytes 34.7 Gbits/sec 0.002 ms 0/132545 (0%)
[ 5] 6.00-7.00 sec 4.03 GBytes 34.6 Gbits/sec 0.003 ms 0/132054 (0%)
[ 5] 7.00-8.00 sec 4.02 GBytes 34.5 Gbits/sec 0.003 ms 0/131631 (0%)
[ 5] 8.00-9.00 sec 4.03 GBytes 34.6 Gbits/sec 0.003 ms 0/132085 (0%)
[ 5] 9.00-10.00 sec 4.00 GBytes 34.4 Gbits/sec 0.002 ms 1296/132380 (0.98%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 40.4 GBytes 34.7 Gbits/sec 0.000 ms 0/1323608 (0%) sender
[ 5] 0.00-10.00 sec 40.4 GBytes 34.7 Gbits/sec 0.002 ms 1296/1323604 (0.098%) receiver
iperf Done.
GL.iNet Beryl AX (GL-MT3000)
root@OpenWrt:~# ubus call system board
{
"kernel": "6.6.52",
"hostname": "OpenWrt",
"system": "ARMv8 Processor rev 4",
"model": "GL.iNet GL-MT3000",
"board_name": "glinet,gl-mt3000",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "SNAPSHOT",
"revision": "r27466-f368e2d5ec",
"target": "mediatek/filogic",
"description": "OpenWrt SNAPSHOT r27466-f368e2d5ec"
}
}
BusyBox v1.36.1 (2024-09-18 08:56:14 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r27466-f368e2d5ec
-----------------------------------------------------
root@OpenWrt:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 49458 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 485 MBytes 4.07 Gbits/sec 0 2.37 MBytes
[ 5] 1.00-2.00 sec 481 MBytes 4.04 Gbits/sec 0 2.37 MBytes
[ 5] 2.00-3.00 sec 474 MBytes 3.97 Gbits/sec 0 3.93 MBytes
[ 5] 3.00-4.00 sec 489 MBytes 4.10 Gbits/sec 0 3.93 MBytes
[ 5] 4.00-5.00 sec 488 MBytes 4.09 Gbits/sec 0 3.93 MBytes
[ 5] 5.00-6.00 sec 492 MBytes 4.13 Gbits/sec 0 3.93 MBytes
[ 5] 6.00-7.00 sec 480 MBytes 4.02 Gbits/sec 0 3.93 MBytes
[ 5] 7.00-8.00 sec 491 MBytes 4.12 Gbits/sec 0 3.93 MBytes
[ 5] 8.00-9.00 sec 490 MBytes 4.11 Gbits/sec 0 3.93 MBytes
[ 5] 9.00-10.00 sec 476 MBytes 3.99 Gbits/sec 0 3.93 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 4.75 GBytes 4.08 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 4.75 GBytes 4.08 Gbits/sec receiver
iperf Done.
TP-Link Archer C7 v4
OpenWRT 23.05
root@OpenWrt:~# ubus call system board
{
"kernel": "5.15.150",
"hostname": "OpenWrt",
"system": "Qualcomm Atheros QCA956X ver 1 rev 0",
"model": "TP-Link Archer C7 v4",
"board_name": "tplink,archer-c7-v4",
"rootfs_type": "squashfs",
"release": {
"distribution": "OpenWrt",
"version": "23.05.3",
"revision": "r23809-234f1a2efa",
"target": "ath79/generic",
"description": "OpenWrt 23.05.3 r23809-234f1a2efa"
}
}
root@OpenWrt:~# iperf3 -s -D && iperf3 -c 127.0.0.1
Connecting to host 127.0.0.1, port 5201
[ 5] local 127.0.0.1 port 49966 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 40.8 MBytes 342 Mbits/sec 0 1.06 MBytes
[ 5] 1.00-2.00 sec 42.4 MBytes 355 Mbits/sec 1 1.06 MBytes
[ 5] 2.00-3.00 sec 41.1 MBytes 345 Mbits/sec 0 1.06 MBytes
[ 5] 3.00-4.00 sec 40.4 MBytes 338 Mbits/sec 0 1.06 MBytes
[ 5] 4.00-5.00 sec 40.4 MBytes 339 Mbits/sec 1 1.06 MBytes
[ 5] 5.00-6.00 sec 40.4 MBytes 339 Mbits/sec 0 1.06 MBytes
[ 5] 6.00-7.00 sec 40.0 MBytes 336 Mbits/sec 0 1.06 MBytes
[ 5] 7.00-8.00 sec 40.8 MBytes 342 Mbits/sec 0 1.06 MBytes
[ 5] 8.00-9.00 sec 41.6 MBytes 349 Mbits/sec 0 1.06 MBytes
[ 5] 9.00-10.00 sec 40.8 MBytes 342 Mbits/sec 1 1.06 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 408 MBytes 343 Mbits/sec 3 sender
[ 5] 0.00-10.00 sec 407 MBytes 342 Mbits/sec receiver
iperf Done.
root@OpenWrt:~# iperf3 -s -D && iperf3 -c 127.0.0.1 -u -b 100G -R && killall iperf3
Connecting to host 127.0.0.1, port 5201
Reverse mode, remote host 127.0.0.1 is sending
[ 5] local 127.0.0.1 port 49391 connected to 127.0.0.1 port 5201
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 30.5 MBytes 256 Mbits/sec 0.108 ms 3/981 (0.31%)
[ 5] 1.00-2.00 sec 30.2 MBytes 254 Mbits/sec 0.067 ms 0/968 (0%)
[ 5] 2.00-3.00 sec 30.7 MBytes 258 Mbits/sec 0.181 ms 0/984 (0%)
[ 5] 3.00-4.00 sec 30.1 MBytes 252 Mbits/sec 0.141 ms 0/963 (0%)
[ 5] 4.00-5.00 sec 30.8 MBytes 258 Mbits/sec 0.094 ms 0/985 (0%)
[ 5] 5.00-6.00 sec 31.1 MBytes 261 Mbits/sec 0.118 ms 0/995 (0%)
[ 5] 6.00-7.00 sec 30.8 MBytes 258 Mbits/sec 0.080 ms 0/986 (0%)
[ 5] 7.00-8.00 sec 29.7 MBytes 249 Mbits/sec 0.177 ms 0/951 (0%)
[ 5] 8.00-9.00 sec 29.8 MBytes 250 Mbits/sec 0.147 ms 0/954 (0%)
[ 5] 9.00-10.00 sec 30.5 MBytes 256 Mbits/sec 0.157 ms 0/976 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 304 MBytes 255 Mbits/sec 0.000 ms 0/9743 (0%) sender
[ 5] 0.00-10.00 sec 304 MBytes 255 Mbits/sec 0.157 ms 3/9743 (0.031%) receiver