OpenWrt support for Linksys MX8500

For Developers
426

Hey all, I thought I would do a small update since 25.12 is now officially released.

I’m seeing all-around better performance with my MX8500s.

Devices (all running Linux):

laptop NIC: Marvell/Aquantia AQC107S 10GbE (updated FW to 121a)
desktop NIC: Intel AX210 WiFi 6E 2x2 MIMO
router: WAX206 with Realtek RTL8221B 2.5GbE
AP: MX8500 with Marvell/Aquantia AQR114C 5GbE
switch: Trendnet TEG-S380 8x 2.5 GbE

[MX8500]<6GHz 160MHz>[AX210]

#MX8500 is running iperf3 server, desktop is iperf3 client

$ iperf3 -c 192.168.0.2
Connecting to host 192.168.0.2, port 5201
[  5] local 192.168.0.15 port 53156 connected to 192.168.0.2 port 5201     
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.33 GBytes  1.14 Gbits/sec    0            sender
[  5]   0.00-10.01  sec  1.33 GBytes  1.14 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.2 -R
Connecting to host 192.168.0.2, port 5201
Reverse mode, remote host 192.168.0.2 is sending
[  5] local 192.168.0.15 port 47404 connected to 192.168.0.2 port 5201              
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   909 MBytes   762 Mbits/sec    0            sender
[  5]   0.00-10.00  sec   905 MBytes   759 Mbits/sec                  receiver

$ iperf3 -c 192.168.0.2 --bidir
Connecting to host 192.168.0.2, port 5201
[  5] local 192.168.0.15 port 39386 connected to 192.168.0.2 port 5201
[  7] local 192.168.0.15 port 39402 connected to 192.168.0.2 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec   105 MBytes   879 Mbits/sec    0   3.38 MBytes       
[  7][RX-C]   0.00-1.00   sec  36.6 MBytes   307 Mbits/sec                  
[  5][TX-C]   1.00-2.00   sec   128 MBytes  1.07 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   1.00-2.00   sec  21.2 MBytes   178 Mbits/sec                  
[  5][TX-C]   2.00-3.00   sec   122 MBytes  1.02 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   2.00-3.00   sec  19.5 MBytes   164 Mbits/sec                  
[  5][TX-C]   3.00-4.00   sec   123 MBytes  1.03 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   3.00-4.00   sec  21.4 MBytes   179 Mbits/sec                  
[  5][TX-C]   4.00-5.00   sec   117 MBytes   980 Mbits/sec    0   3.38 MBytes       
[  7][RX-C]   4.00-5.00   sec  20.8 MBytes   174 Mbits/sec                  
[  5][TX-C]   5.00-6.00   sec   118 MBytes   990 Mbits/sec    0   3.38 MBytes       
[  7][RX-C]   5.00-6.00   sec  23.8 MBytes   199 Mbits/sec                  
[  5][TX-C]   6.00-7.00   sec   123 MBytes  1.03 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   6.00-7.00   sec  30.0 MBytes   252 Mbits/sec                  
[  5][TX-C]   7.00-8.00   sec   119 MBytes  1.00 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   7.00-8.00   sec  32.5 MBytes   272 Mbits/sec                  
[  5][TX-C]   8.00-9.00   sec   118 MBytes   994 Mbits/sec    0   3.38 MBytes       
[  7][RX-C]   8.00-9.00   sec  24.4 MBytes   205 Mbits/sec                  
[  5][TX-C]   9.00-10.00  sec   122 MBytes  1.02 Gbits/sec    0   3.38 MBytes       
[  7][RX-C]   9.00-10.00  sec  29.1 MBytes   244 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  1.17 GBytes  1.00 Gbits/sec    0            sender
[  5][TX-C]   0.00-10.01  sec  1.17 GBytes  1.00 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec   263 MBytes   221 Mbits/sec    0            sender
[  7][RX-C]   0.00-10.01  sec   259 MBytes   217 Mbits/sec                  receiver

WiFi performance looks good, no dropped packets but throughput is limited by the MX8500 acting as the iperf3 server, as we’ll see in the next test.

[WAX206 2.5GbE]< cat6 >[TEG-S380 2.5GbE]< cat7 >[MX8500 5GbE]<6GHz 160MHz>[AX210]

#WAX206 is running iperf3 server, desktop is iperf3 client

$ iperf3 -c 192.168.0.1
Connecting to host 192.168.0.1, port 5201
[  5] local 192.168.0.15 port 41998 connected to 192.168.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   153 MBytes  1.28 Gbits/sec    0   2.30 MBytes       
[  5]   1.00-2.00   sec   147 MBytes  1.23 Gbits/sec    0   2.30 MBytes       
[  5]   2.00-3.00   sec   145 MBytes  1.21 Gbits/sec   28   1.64 MBytes       
[  5]   3.00-4.00   sec   146 MBytes  1.22 Gbits/sec    0   1.81 MBytes       
[  5]   4.00-5.00   sec   150 MBytes  1.26 Gbits/sec    0   1.95 MBytes       
[  5]   5.00-6.00   sec   149 MBytes  1.25 Gbits/sec    0   2.06 MBytes       
[  5]   6.00-7.00   sec   149 MBytes  1.25 Gbits/sec    0   2.14 MBytes       
[  5]   7.00-8.00   sec   148 MBytes  1.24 Gbits/sec    0   2.20 MBytes       
[  5]   8.00-9.00   sec   145 MBytes  1.21 Gbits/sec    0   2.21 MBytes       
[  5]   9.00-10.00  sec   148 MBytes  1.24 Gbits/sec    0   2.21 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.45 GBytes  1.24 Gbits/sec   28            sender
[  5]   0.00-10.01  sec  1.45 GBytes  1.24 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.1 -R
Connecting to host 192.168.0.1, port 5201
Reverse mode, remote host 192.168.0.1 is sending
[  5] local 192.168.0.15 port 60888 connected to 192.168.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec  1.77 GBytes  1.52 Gbits/sec    0            sender
[  5]   0.00-10.00  sec  1.76 GBytes  1.52 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.1 --bidir
Connecting to host 192.168.0.1, port 5201
[  5] local 192.168.0.15 port 35474 connected to 192.168.0.1 port 5201
[  7] local 192.168.0.15 port 35486 connected to 192.168.0.1 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec  72.4 MBytes   607 Mbits/sec    0   2.27 MBytes       
[  7][RX-C]   0.00-1.00   sec   101 MBytes   848 Mbits/sec                  
[  5][TX-C]   1.00-2.00   sec  73.4 MBytes   615 Mbits/sec    0   2.27 MBytes       
[  7][RX-C]   1.00-2.00   sec   112 MBytes   941 Mbits/sec                  
[  5][TX-C]   2.00-3.00   sec  67.9 MBytes   570 Mbits/sec   16   1.75 MBytes       
[  7][RX-C]   2.00-3.00   sec   126 MBytes  1.06 Gbits/sec                  
[  5][TX-C]   3.00-4.00   sec  69.2 MBytes   581 Mbits/sec    0   1.90 MBytes       
[  7][RX-C]   3.00-4.00   sec   119 MBytes  1.00 Gbits/sec                  
[  5][TX-C]   4.00-5.00   sec  72.6 MBytes   610 Mbits/sec    0   2.01 MBytes       
[  7][RX-C]   4.00-5.00   sec   113 MBytes   945 Mbits/sec                  
[  5][TX-C]   5.00-6.00   sec  71.8 MBytes   602 Mbits/sec    0   2.10 MBytes       
[  7][RX-C]   5.00-6.00   sec   118 MBytes   988 Mbits/sec                  
[  5][TX-C]   6.00-7.00   sec  74.2 MBytes   622 Mbits/sec    0   2.17 MBytes       
[  7][RX-C]   6.00-7.00   sec   117 MBytes   983 Mbits/sec                  
[  5][TX-C]   7.00-8.00   sec  67.2 MBytes   564 Mbits/sec   53   1.61 MBytes       
[  7][RX-C]   7.00-8.00   sec   127 MBytes  1.07 Gbits/sec                  
[  5][TX-C]   8.00-9.00   sec  68.2 MBytes   573 Mbits/sec    0   1.69 MBytes       
[  7][RX-C]   8.00-9.00   sec   124 MBytes  1.04 Gbits/sec                  
[  5][TX-C]   9.00-10.00  sec  62.0 MBytes   520 Mbits/sec   22   1.25 MBytes       
[  7][RX-C]   9.00-10.00  sec   126 MBytes  1.06 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec   699 MBytes   586 Mbits/sec   91            sender
[  5][TX-C]   0.00-10.01  sec   697 MBytes   584 Mbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  1.16 GBytes   996 Mbits/sec    0            sender
[  7][RX-C]   0.00-10.01  sec  1.16 GBytes   992 Mbits/sec                  receiver

The MX8500’s 5GbE port has gotten better over time, but still isn’t capable of reliable 5GbE or even fully saturated 2.5GbE.

[MX8500 5GbE]< cat7 >[TEG-S380 2.5GbE]< cat7 >[AQC107S 10GbE]

#MX8500 is running iperf3 server, laptop is iperf3 client

$ iperf3 -c 192.168.0.2
Connecting to host 192.168.0.2, port 5201
[  5] local 192.168.0.14 port 53790 connected to 192.168.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.69 GBytes  2.31 Gbits/sec    0            sender
[  5]   0.00-10.00  sec  2.68 GBytes  2.30 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.2 -R
Connecting to host 192.168.0.2, port 5201
Reverse mode, remote host 192.168.0.2 is sending
[  5] local 192.168.0.14 port 37984 connected to 192.168.0.2 port 5201
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.26 GBytes  1.94 Gbits/sec    0            sender
[  5]   0.00-10.00  sec  2.26 GBytes  1.94 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.2 --bidir
Connecting to host 192.168.0.2, port 5201
[  5] local 192.168.0.14 port 37700 connected to 192.168.0.2 port 5201
[  7] local 192.168.0.14 port 37710 connected to 192.168.0.2 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec   202 MBytes  1.69 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   0.00-1.00   sec   164 MBytes  1.37 Gbits/sec                  
[  5][TX-C]   1.00-2.00   sec   193 MBytes  1.62 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   1.00-2.00   sec   171 MBytes  1.44 Gbits/sec                  
[  5][TX-C]   2.00-3.00   sec   192 MBytes  1.61 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   2.00-3.00   sec   171 MBytes  1.43 Gbits/sec                  
[  5][TX-C]   3.00-4.00   sec   191 MBytes  1.60 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   3.00-4.00   sec   170 MBytes  1.43 Gbits/sec                  
[  5][TX-C]   4.00-5.00   sec   190 MBytes  1.60 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   4.00-5.00   sec   170 MBytes  1.42 Gbits/sec                  
[  5][TX-C]   5.00-6.00   sec   188 MBytes  1.57 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   5.00-6.00   sec   170 MBytes  1.43 Gbits/sec                  
[  5][TX-C]   6.00-7.00   sec   189 MBytes  1.59 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   6.00-7.00   sec   170 MBytes  1.43 Gbits/sec                  
[  5][TX-C]   7.00-8.00   sec   186 MBytes  1.56 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   7.00-8.00   sec   170 MBytes  1.42 Gbits/sec                  
[  5][TX-C]   8.00-9.00   sec   184 MBytes  1.55 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   8.00-9.00   sec   170 MBytes  1.42 Gbits/sec                  
[  5][TX-C]   9.00-10.00  sec   185 MBytes  1.55 Gbits/sec    0   1.33 MBytes       
[  7][RX-C]   9.00-10.00  sec   169 MBytes  1.42 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  1.85 GBytes  1.59 Gbits/sec    0            sender
[  5][TX-C]   0.00-10.00  sec  1.85 GBytes  1.59 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  1.66 GBytes  1.42 Gbits/sec    0            sender
[  7][RX-C]   0.00-10.00  sec  1.65 GBytes  1.42 Gbits/sec                  receiver

As we can see, the 5GbE port still isn’t optimized yet.

Here’s the base line performance of the AQC107S with the WAX206 acting as server.

[WAX206 2.5GbE]< cat6 >[TEG-S380 2.5GbE]< cat7 >[AQC107S 10GbE]

#WAX206 is running iperf3 server, laptop is iperf3 client

$ iperf3 -c 192.168.0.1
Connecting to host 192.168.0.1, port 5201
[  5] local 192.168.0.14 port 45958 connected to 192.168.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.69 GBytes  2.31 Gbits/sec    0            sender
[  5]   0.00-10.00  sec  2.68 GBytes  2.30 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.1 -R
Connecting to host 192.168.0.1, port 5201
Reverse mode, remote host 192.168.0.1 is sending
[  5] local 192.168.0.14 port 58654 connected to 192.168.0.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.73 GBytes  2.35 Gbits/sec    0            sender
[  5]   0.00-10.00  sec  2.73 GBytes  2.34 Gbits/sec                  receiver

$ iperf3 -c 192.168.0.1 --bidir
Connecting to host 192.168.0.1, port 5201
[  5] local 192.168.0.14 port 50868 connected to 192.168.0.1 port 5201
[  7] local 192.168.0.14 port 50870 connected to 192.168.0.1 port 5201
[ ID][Role] Interval           Transfer     Bitrate         Retr  Cwnd
[  5][TX-C]   0.00-1.00   sec   207 MBytes  1.74 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   0.00-1.00   sec   252 MBytes  2.11 Gbits/sec                  
[  5][TX-C]   1.00-2.00   sec   244 MBytes  2.05 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   1.00-2.00   sec   258 MBytes  2.16 Gbits/sec                  
[  5][TX-C]   2.00-3.00   sec   244 MBytes  2.05 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   2.00-3.00   sec   251 MBytes  2.11 Gbits/sec                  
[  5][TX-C]   3.00-4.00   sec   253 MBytes  2.12 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   3.00-4.00   sec   255 MBytes  2.14 Gbits/sec                  
[  5][TX-C]   4.00-5.00   sec   248 MBytes  2.08 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   4.00-5.00   sec   255 MBytes  2.14 Gbits/sec                  
[  5][TX-C]   5.00-6.00   sec   232 MBytes  1.94 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   5.00-6.00   sec   262 MBytes  2.20 Gbits/sec                  
[  5][TX-C]   6.00-7.00   sec   240 MBytes  2.01 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   6.00-7.00   sec   264 MBytes  2.21 Gbits/sec                  
[  5][TX-C]   7.00-8.00   sec   207 MBytes  1.74 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   7.00-8.00   sec   260 MBytes  2.18 Gbits/sec                  
[  5][TX-C]   8.00-9.00   sec   203 MBytes  1.70 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   8.00-9.00   sec   263 MBytes  2.21 Gbits/sec                  
[  5][TX-C]   9.00-10.00  sec   206 MBytes  1.73 Gbits/sec    0   2.22 MBytes       
[  7][RX-C]   9.00-10.00  sec   265 MBytes  2.22 Gbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID][Role] Interval           Transfer     Bitrate         Retr
[  5][TX-C]   0.00-10.00  sec  2.23 GBytes  1.92 Gbits/sec    0            sender
[  5][TX-C]   0.00-10.01  sec  2.23 GBytes  1.91 Gbits/sec                  receiver
[  7][RX-C]   0.00-10.00  sec  2.53 GBytes  2.17 Gbits/sec    0            sender
[  7][RX-C]   0.00-10.01  sec  2.52 GBytes  2.17 Gbits/sec                  receiver
427

I reran the same iperf3 test between two MX8500s now that I updated both to 25.12. Performance seems better, but still nowhere near 5GbE. I also updated my switch that sits in between the two mx8500s though, so maybe that was a factor as well.

$ iperf3 -c 192.168.10.8
Connecting to host 192.168.10.8, port 5201
[  5] local 192.168.10.3 port 34936 connected to 192.168.10.8 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   216 MBytes  1.81 Gbits/sec   66    305 KBytes
[  5]   1.00-2.00   sec   215 MBytes  1.80 Gbits/sec   29    485 KBytes
[  5]   2.00-3.00   sec   214 MBytes  1.79 Gbits/sec   46    272 KBytes
[  5]   3.00-4.00   sec   213 MBytes  1.79 Gbits/sec   45    426 KBytes
[  5]   4.00-5.00   sec   210 MBytes  1.76 Gbits/sec  186    303 KBytes
[  5]   5.00-6.00   sec   209 MBytes  1.76 Gbits/sec    1    503 KBytes
[  5]   6.00-7.00   sec   214 MBytes  1.79 Gbits/sec  130    338 KBytes
[  5]   7.00-8.00   sec   212 MBytes  1.78 Gbits/sec   98    433 KBytes
[  5]   8.00-9.00   sec   214 MBytes  1.80 Gbits/sec  120    280 KBytes
[  5]   9.00-10.00  sec   212 MBytes  1.78 Gbits/sec   50    305 KBytes




[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  2.09 GBytes  1.80 Gbits/sec  771            sender
[  5]   0.00-10.00  sec  2.09 GBytes  1.80 Gbits/sec                  receiver

iperf Done.
428

Have you tested builds with the new driver: https://github.com/openwrt/openwrt/pull/22381 ?

I was able to get better results: https://github.com/openwrt/openwrt/pull/22381#issuecomment-4177963191

429

no i haven’t, but good to know this is coming.