I got 4 used MX8500s for free recently and have installed OpenWRT 24.10.3 and updated the AQR FW to 5.6.5 (was 5.4.b). Iām actually kinda sad that there are no client devices that use the ānewā UNII-4 5GHz bands 159, 163 and 167 (Intelās AX200 and AX210 show them as disabled with iw list). But I guess these can still be used for a backhaul between APs, which is better than nothing I suppose.
Anyways, I was doing some iperf3 tests with the 5GbE port and I get worse performance between two MX8500s with a 5ft shielded CAT6 cable than I do with a WAX206 that has a 2.5GbE port. Furthermore, I am seeing random packet loss on the MX8500s 5GbE port⦠I never did any testing with the default Linksys FW (looked awful so I nuked it from orbit with OpenWRT when I got these) so I donāt know if this is just an unfortunate quirk of this platform or if there is possible room for improvement.
Test setup: TRENDnet TEG-S308/A 8-port 2.5GbE switch with 4x MX8500 connected with 5GbE port, 1x WAX206 connected with 2.5GbE port and 1x Lenovo TB4 Dock with (I225v3) 2.5GbE port.
Baseline: Laptop I225v3 via 25ft of CAT5E and WAX206 via 4ft of CAT5E to 2.5GbE switch:
iperf3 -c 192.168.1.19 #WAX206 to I225v3
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.74 GBytes 2.35 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec receiver
iperf3 -c 192.168.1.19 -R #I225v3 to WAX206
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 2.52 GBytes 2.16 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec receiver
iperf3 -c 192.168.1.19 --bidir
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 2.58 GBytes 2.21 Gbits/sec 0 sender
[ 5][TX-C] 0.00-10.01 sec 2.57 GBytes 2.21 Gbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.84 GBytes 1.58 Gbits/sec 0 sender
[ 7][RX-C] 0.00-10.01 sec 1.83 GBytes 1.57 Gbits/sec receiver
The WAX206 has higher throughput than the Intel I225v3. No packet loss in either direction. Cables seem fine for 2.5GbE use.
WAX206 and MX8500 #1 via 5ft CAT6 STP cable to switch.
iperf3 -c 192.168.1.7 #WAX206 to MX8500
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.73 GBytes 2.35 Gbits/sec 0 sender
[ 5] 0.00-10.01 sec 2.73 GBytes 2.34 Gbits/sec receiver
iperf3 -c 192.168.1.7 -R #MX8500 to WAX206
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.01 sec 2.49 GBytes 2.14 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.49 GBytes 2.14 Gbits/sec receiver
iperf3 -c 192.168.1.7 --bidir
[ 5][TX-C] 0.00-10.00 sec 1.43 GBytes 1.23 Gbits/sec 0 sender
[ 5][TX-C] 0.00-10.01 sec 1.43 GBytes 1.22 Gbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 885 MBytes 742 Mbits/sec 0 sender
[ 7][RX-C] 0.00-10.01 sec 882 MBytes 739 Mbits/sec receiver
The MX8500ās 5GbE port looks okay initially but craps out with --bidir. No packet lossā¦yet. If I run the receive mode enough times I do eventually get packet loss with the MX8500 as transmitter while the WAX206 transmitting and MX8500 receiving doesnāt show any loss.
Hereās the same tests with MX8500 #1 and the Laptop I225v3 as server.
iperf3 -c 192.168.1.19 #MX8500 to I225v3
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.51 GBytes 2.16 Gbits/sec 7 sender
[ 5] 0.00-10.00 sec 2.52 GBytes 2.16 Gbits/sec receiver
iperf3 -c 192.168.1.19 -R #I225v3 to MX8500
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.52 GBytes 2.17 Gbits/sec 0 sender
[ 5] 0.00-10.00 sec 2.52 GBytes 2.16 Gbits/sec receiver
iperf3 -c 192.168.1.19 --bidir
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.71 GBytes 1.47 Gbits/sec 0 sender
[ 5][TX-C] 0.00-10.00 sec 1.71 GBytes 1.47 Gbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.44 GBytes 1.24 Gbits/sec 0 sender
[ 7][RX-C] 0.00-10.00 sec 1.44 GBytes 1.24 Gbits/sec receiver
Slightly better performance with --bidir, but we now have packet loss. The retransmits can be as high as 500 in some tests but jumps around from 0 to less than 600 or so.
And now two MX8500s (#1 and #2) each connected via 5ft CAT6 to the switch
iperf3 -c 192.168.1.7 #MX8500 #2 to MX8500 #1
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.88 GBytes 1.61 Gbits/sec 1084 sender
[ 5] 0.00-10.00 sec 1.87 GBytes 1.61 Gbits/sec receiver
iperf3 -c 192.168.1.7 -R #MX8500 #1 to MX8500 #2
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.30 GBytes 1.98 Gbits/sec 1209 sender
[ 5] 0.00-10.00 sec 2.30 GBytes 1.97 Gbits/sec receiver
iperf3 -c 192.168.1.7 --bidir
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.12 GBytes 962 Mbits/sec 5 sender
[ 5][TX-C] 0.00-10.00 sec 1.12 GBytes 960 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 964 MBytes 808 Mbits/sec 94 sender
[ 7][RX-C] 0.00-10.00 sec 961 MBytes 806 Mbits/sec receiver
Lots of packet lossā¦and pretty bad performance vs. WAX206 and the Intel I225v3 NICs.
And here is when both MX8500 are connected to each other directly via the 5GbE ports with 5ft CAT6 (1GbE port on MX8500 #1 is connected to switch for ssh access).
iperf3 -c 192.168.1.7 #MX8500 #2 to MX8500 #1
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.21 GBytes 1.90 Gbits/sec 175 sender
[ 5] 0.00-10.01 sec 2.21 GBytes 1.89 Gbits/sec receiver
iperf3 -c 192.168.1.7 -R #MX8500 #1 to MX8500 #2
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 2.12 GBytes 1.82 Gbits/sec 156 sender
[ 5] 0.00-10.00 sec 2.11 GBytes 1.82 Gbits/sec receiver
iperf3 -c 192.168.1.7 --bidir
[ 5][TX-C] 0.00-10.00 sec 1.07 GBytes 921 Mbits/sec 21 sender
[ 5][TX-C] 0.00-10.00 sec 1.07 GBytes 918 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 976 MBytes 819 Mbits/sec 67 sender
[ 7][RX-C] 0.00-10.00 sec 973 MBytes 816 Mbits/sec receiver
Still packet loss. Clearly, the 2.5GbE switch is not the issue. Changing Ethernet cables does not help.
And for fun, here is MX8500#2 <-5GbE-> MX8500#1 <-1GbE-> switch <-2.5GbE-> I225v3
iperf3 -c 192.168.1.19 #MX8500 #2 to I225v3
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 942 Mbits/sec 199 sender
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver
iperf3 -c 192.168.1.19 -R #I225v3 to MX8500 #2
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 944 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec receiver
iperf3 -c 192.168.1.19 --bidir
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 860 MBytes 721 Mbits/sec 262 sender
[ 5][TX-C] 0.00-10.00 sec 858 MBytes 720 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 933 MBytes 783 Mbits/sec 485 sender
[ 7][RX-C] 0.00-10.00 sec 929 MBytes 779 Mbits/sec receiver
Lots of packet loss. Clearly, these 5GbE ports have issues.
And, for completeness, MX8500#2 <-1GbE-> switch <-2.5GbE-> I225v3
iperf3 -c 192.168.1.19
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 943 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec
iperf3 -c 192.168.1.19 -R
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 1.10 GBytes 945 Mbits/sec 0 sender
[ 5] 0.00-10.00 sec 1.10 GBytes 941 Mbits/sec
iperf3 -c 192.168.1.19 --bidir
[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 1.08 GBytes 931 Mbits/sec 0 sender
[ 5][TX-C] 0.00-10.00 sec 1.08 GBytes 929 Mbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 1.06 GBytes 914 Mbits/sec 0 sender
[ 7][RX-C] 0.00-10.00 sec 1.06 GBytes 911 Mbits/sec
The 1GbE ports work great, no packet loss and perf seems good, and --bidir looks great compared to the MX8500 5GbE to 5GbE or 5GbE to 2.5GbE switch perf.
Iām at a loss for what to do here⦠I guess It is probably best not to use the 5GbE port for anything with OpenWRT installed. They just donāt seem reliable even with newer 5.6.5 firmware.