Low speeds on IPv6 with SQM Cake

huaracheguarache · October 1, 2018, 2:03pm

My ISP provides IPv6 through 6rd, and to use that I have added option iface6rd '1' under my wan interface in the network config. This automatically creates a virtual dynamic interface from the 6rd settings my ISP provides.

I'm running OpenWrt SNAPSHOT r8175-fcfb9e4ded, and there seems to be an issue with how SQM Cake handles the 6rd traffic. Browsing the web with SQM Cake and 6rd enabled is not a very good experience since IPv6 websites tend to load slowly. I ran some iperf3 tests to compare the speeds IPv4 and IPv6 traffic get on different SQM disciplines and scripts. All tests are set to 100 mbps download and 100 mbps upload.

cake/layer_cake:

IPv6

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 60700 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  69.6 KBytes   570 Kbits/sec                  
[  5]   1.00-2.00   sec  29.5 KBytes   242 Kbits/sec                  
[  5]   2.00-3.00   sec  38.9 KBytes   319 Kbits/sec                  
[  5]   3.00-4.00   sec  47.2 KBytes   387 Kbits/sec                  
[  5]   4.00-5.00   sec  53.1 KBytes   435 Kbits/sec                  
[  5]   5.00-6.00   sec  51.9 KBytes   425 Kbits/sec                  
[  5]   6.00-7.00   sec  62.5 KBytes   512 Kbits/sec                  
[  5]   7.00-8.00   sec  50.7 KBytes   416 Kbits/sec                  
[  5]   8.00-9.00   sec  43.6 KBytes   358 Kbits/sec                  
[  5]   9.00-10.00  sec  59.0 KBytes   483 Kbits/sec                  
[  5]  10.00-11.00  sec  54.3 KBytes   445 Kbits/sec                  
[  5]  11.00-12.00  sec  83.8 KBytes   686 Kbits/sec                  
[  5]  12.00-13.00  sec  64.9 KBytes   532 Kbits/sec                  
[  5]  13.00-14.00  sec  59.0 KBytes   483 Kbits/sec                  
[  5]  14.00-15.00  sec  73.1 KBytes   599 Kbits/sec                  
[  5]  15.00-16.00  sec  53.1 KBytes   435 Kbits/sec                  
[  5]  16.00-17.00  sec  44.8 KBytes   367 Kbits/sec                  
[  5]  17.00-18.00  sec  37.8 KBytes   309 Kbits/sec                  
[  5]  18.00-19.00  sec  49.5 KBytes   406 Kbits/sec                  
[  5]  19.00-20.00  sec  69.6 KBytes   570 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec  1.11 MBytes   464 Kbits/sec  262             sender
[  5]   0.00-20.00  sec  1.07 MBytes   449 Kbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 2a01:79c:cebd:1e2c::a80 port 60704 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  7.12 MBytes  59.7 Mbits/sec    1    348 KBytes       
[  5]   1.00-2.00   sec  7.94 MBytes  66.6 Mbits/sec    0    393 KBytes       
[  5]   2.00-3.00   sec  7.33 MBytes  61.5 Mbits/sec    0    424 KBytes       
[  5]   3.00-4.00   sec  7.94 MBytes  66.6 Mbits/sec    0    444 KBytes       
[  5]   4.00-5.00   sec  7.45 MBytes  62.5 Mbits/sec    0    453 KBytes       
[  5]   5.00-6.00   sec  7.51 MBytes  63.0 Mbits/sec    0    457 KBytes       
[  5]   6.00-7.00   sec  8.00 MBytes  67.1 Mbits/sec    0    457 KBytes       
[  5]   7.00-8.00   sec  7.33 MBytes  61.5 Mbits/sec    0    457 KBytes       
[  5]   8.00-9.00   sec  7.94 MBytes  66.6 Mbits/sec    0    459 KBytes       
[  5]   9.00-10.00  sec  7.39 MBytes  62.0 Mbits/sec    0    467 KBytes       
[  5]  10.00-11.00  sec  8.00 MBytes  67.1 Mbits/sec    0    484 KBytes       
[  5]  11.00-12.00  sec  7.82 MBytes  65.6 Mbits/sec    0    510 KBytes       
[  5]  12.00-13.00  sec  7.82 MBytes  65.6 Mbits/sec    0    550 KBytes       
[  5]  13.00-14.00  sec  7.57 MBytes  63.5 Mbits/sec    0    606 KBytes       
[  5]  14.00-15.00  sec  8.49 MBytes  71.2 Mbits/sec    0    683 KBytes       
[  5]  15.00-16.00  sec  7.51 MBytes  63.0 Mbits/sec    0    780 KBytes       
[  5]  16.00-17.00  sec  8.49 MBytes  71.2 Mbits/sec    0    901 KBytes       
[  5]  17.00-18.00  sec  8.57 MBytes  71.9 Mbits/sec    0   1.03 MBytes       
[  5]  18.00-19.00  sec  7.50 MBytes  62.9 Mbits/sec    0   1.20 MBytes       
[  5]  19.00-20.00  sec  7.50 MBytes  62.9 Mbits/sec    0   1.40 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   155 MBytes  65.1 Mbits/sec    1             sender
[  5]   0.00-20.00  sec   154 MBytes  64.6 Mbits/sec                  receiver

iperf Done.

IPv4

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 192.168.1.192 port 43892 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  1.94 MBytes  16.3 Mbits/sec                  
[  5]   1.00-2.00   sec  2.54 MBytes  21.3 Mbits/sec                  
[  5]   2.00-3.00   sec  2.80 MBytes  23.5 Mbits/sec                  
[  5]   3.00-4.00   sec  3.26 MBytes  27.3 Mbits/sec                  
[  5]   4.00-5.00   sec  3.66 MBytes  30.7 Mbits/sec                  
[  5]   5.00-6.00   sec  4.15 MBytes  34.8 Mbits/sec                  
[  5]   6.00-7.00   sec  4.73 MBytes  39.7 Mbits/sec                  
[  5]   7.00-8.00   sec  5.04 MBytes  42.3 Mbits/sec                  
[  5]   8.00-9.00   sec  5.52 MBytes  46.3 Mbits/sec                  
[  5]   9.00-10.00  sec  5.98 MBytes  50.2 Mbits/sec                  
[  5]  10.00-11.00  sec  6.44 MBytes  54.0 Mbits/sec                  
[  5]  11.00-12.00  sec  8.04 MBytes  67.5 Mbits/sec                  
[  5]  12.00-13.00  sec  10.3 MBytes  86.5 Mbits/sec                  
[  5]  13.00-14.00  sec  10.9 MBytes  91.6 Mbits/sec                  
[  5]  14.00-15.00  sec  10.8 MBytes  90.7 Mbits/sec                  
[  5]  15.00-16.00  sec  11.3 MBytes  94.5 Mbits/sec                  
[  5]  16.00-17.00  sec  10.5 MBytes  87.9 Mbits/sec                  
[  5]  17.00-18.00  sec  10.1 MBytes  84.8 Mbits/sec                  
[  5]  18.00-19.00  sec  10.6 MBytes  88.7 Mbits/sec                  
[  5]  19.00-20.00  sec  10.8 MBytes  90.8 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   142 MBytes  59.5 Mbits/sec    3             sender
[  5]   0.00-20.00  sec   139 MBytes  58.5 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 192.168.1.192 port 43896 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  10.4 MBytes  86.9 Mbits/sec    1    742 KBytes       
[  5]   1.00-2.00   sec  10.0 MBytes  83.9 Mbits/sec    0    836 KBytes       
[  5]   2.00-3.00   sec  8.75 MBytes  73.4 Mbits/sec    0    905 KBytes       
[  5]   3.00-4.00   sec  10.0 MBytes  83.9 Mbits/sec    0    954 KBytes       
[  5]   4.00-5.00   sec  10.0 MBytes  83.9 Mbits/sec    0    988 KBytes       
[  5]   5.00-6.00   sec  10.0 MBytes  83.9 Mbits/sec    0   1008 KBytes       
[  5]   6.00-7.00   sec  10.0 MBytes  83.9 Mbits/sec    0   1020 KBytes       
[  5]   7.00-8.00   sec  10.0 MBytes  83.9 Mbits/sec    0   1022 KBytes       
[  5]   8.00-9.00   sec  10.0 MBytes  83.9 Mbits/sec    0   1022 KBytes       
[  5]   9.00-10.00  sec  8.75 MBytes  73.4 Mbits/sec    0   1022 KBytes       
[  5]  10.00-11.00  sec  10.0 MBytes  83.9 Mbits/sec    1    727 KBytes       
[  5]  11.00-12.00  sec  8.75 MBytes  73.4 Mbits/sec    0    823 KBytes       
[  5]  12.00-13.00  sec  10.0 MBytes  83.9 Mbits/sec    0    897 KBytes       
[  5]  13.00-14.00  sec  10.0 MBytes  83.9 Mbits/sec    0    950 KBytes       
[  5]  14.00-15.00  sec  10.0 MBytes  83.9 Mbits/sec    0    987 KBytes       
[  5]  15.00-16.00  sec  10.0 MBytes  83.9 Mbits/sec    0   1008 KBytes       
[  5]  16.00-17.00  sec  8.75 MBytes  73.4 Mbits/sec    0   1021 KBytes       
[  5]  17.00-18.00  sec  10.0 MBytes  83.9 Mbits/sec    0   1.00 MBytes       
[  5]  18.00-19.00  sec  10.0 MBytes  83.9 Mbits/sec    0   1.00 MBytes       
[  5]  19.00-20.00  sec  10.0 MBytes  83.9 Mbits/sec    0   1.00 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   195 MBytes  81.9 Mbits/sec    2             sender
[  5]   0.00-20.00  sec   193 MBytes  81.2 Mbits/sec                  receiver

iperf Done.

cake/piece_of_cake:

IPv6

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 60724 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  56.6 KBytes   464 Kbits/sec                  
[  5]   1.00-2.00   sec  18.9 KBytes   155 Kbits/sec                  
[  5]   2.00-3.00   sec  17.7 KBytes   145 Kbits/sec                  
[  5]   3.00-4.00   sec  53.1 KBytes   435 Kbits/sec                  
[  5]   4.00-5.00   sec  57.8 KBytes   474 Kbits/sec                  
[  5]   5.00-6.00   sec  33.0 KBytes   271 Kbits/sec                  
[  5]   6.00-7.00   sec  70.8 KBytes   580 Kbits/sec                  
[  5]   7.00-8.00   sec  44.8 KBytes   367 Kbits/sec                  
[  5]   8.00-9.00   sec  42.5 KBytes   348 Kbits/sec                  
[  5]   9.00-10.00  sec  61.3 KBytes   502 Kbits/sec                  
[  5]  10.00-11.00  sec  30.7 KBytes   251 Kbits/sec                  
[  5]  11.00-12.00  sec  59.0 KBytes   483 Kbits/sec                  
[  5]  12.00-13.00  sec  49.5 KBytes   406 Kbits/sec                  
[  5]  13.00-14.00  sec  51.9 KBytes   425 Kbits/sec                  
[  5]  14.00-15.00  sec  24.8 KBytes   203 Kbits/sec                  
[  5]  15.00-16.00  sec  10.6 KBytes  87.0 Kbits/sec                  
[  5]  16.00-17.00  sec  41.3 KBytes   338 Kbits/sec                  
[  5]  17.00-18.00  sec  31.9 KBytes   261 Kbits/sec                  
[  5]  18.00-19.00  sec  29.5 KBytes   242 Kbits/sec                  
[  5]  19.00-20.00  sec  64.9 KBytes   532 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   917 KBytes   375 Kbits/sec  250             sender
[  5]   0.00-20.00  sec   851 KBytes   348 Kbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 2a01:79c:cebd:1e2c::a80 port 60730 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  8.04 MBytes  67.4 Mbits/sec    1    382 KBytes       
[  5]   1.00-2.00   sec  8.06 MBytes  67.6 Mbits/sec    0    433 KBytes       
[  5]   2.00-3.00   sec  8.06 MBytes  67.6 Mbits/sec    0    467 KBytes       
[  5]   3.00-4.00   sec  8.06 MBytes  67.6 Mbits/sec    0    491 KBytes       
[  5]   4.00-5.00   sec  8.73 MBytes  73.2 Mbits/sec    0    504 KBytes       
[  5]   5.00-6.00   sec  7.63 MBytes  64.0 Mbits/sec    0    508 KBytes       
[  5]   6.00-7.00   sec  8.18 MBytes  68.6 Mbits/sec    0    510 KBytes       
[  5]   7.00-8.00   sec  8.18 MBytes  68.6 Mbits/sec    0    510 KBytes       
[  5]   8.00-9.00   sec  8.36 MBytes  70.2 Mbits/sec    0    511 KBytes       
[  5]   9.00-10.00  sec  8.43 MBytes  70.7 Mbits/sec    0    517 KBytes       
[  5]  10.00-11.00  sec  8.06 MBytes  67.6 Mbits/sec    0    531 KBytes       
[  5]  11.00-12.00  sec  7.75 MBytes  65.0 Mbits/sec    0    553 KBytes       
[  5]  12.00-13.00  sec  8.18 MBytes  68.6 Mbits/sec    0    587 KBytes       
[  5]  13.00-14.00  sec  8.73 MBytes  73.2 Mbits/sec    0    639 KBytes       
[  5]  14.00-15.00  sec  7.88 MBytes  66.1 Mbits/sec    0    709 KBytes       
[  5]  15.00-16.00  sec  8.67 MBytes  72.7 Mbits/sec    0    799 KBytes       
[  5]  16.00-17.00  sec  8.85 MBytes  74.3 Mbits/sec    0    911 KBytes       
[  5]  17.00-18.00  sec  8.66 MBytes  72.7 Mbits/sec    0   1.03 MBytes       
[  5]  18.00-19.00  sec  8.75 MBytes  73.4 Mbits/sec    0   1.19 MBytes       
[  5]  19.00-20.00  sec  7.50 MBytes  62.9 Mbits/sec    0   1.38 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   165 MBytes  69.1 Mbits/sec    1             sender
[  5]   0.00-20.00  sec   163 MBytes  68.5 Mbits/sec                  receiver

iperf Done.

IPv4

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 192.168.1.192 port 43998 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.71 MBytes  22.8 Mbits/sec                  
[  5]   1.00-2.00   sec  3.20 MBytes  26.8 Mbits/sec                  
[  5]   2.00-3.00   sec  3.60 MBytes  30.2 Mbits/sec                  
[  5]   3.00-4.00   sec  3.88 MBytes  32.6 Mbits/sec                  
[  5]   4.00-5.00   sec  4.37 MBytes  36.6 Mbits/sec                  
[  5]   5.00-6.00   sec  4.81 MBytes  40.3 Mbits/sec                  
[  5]   6.00-7.00   sec  5.10 MBytes  42.8 Mbits/sec                  
[  5]   7.00-8.00   sec  5.65 MBytes  47.4 Mbits/sec                  
[  5]   8.00-9.00   sec  5.99 MBytes  50.3 Mbits/sec                  
[  5]   9.00-10.00  sec  6.48 MBytes  54.3 Mbits/sec                  
[  5]  10.00-11.00  sec  6.90 MBytes  57.9 Mbits/sec                  
[  5]  11.00-12.00  sec  8.09 MBytes  67.9 Mbits/sec                  
[  5]  12.00-13.00  sec  10.0 MBytes  84.0 Mbits/sec                  
[  5]  13.00-14.00  sec  11.4 MBytes  95.3 Mbits/sec                  
[  5]  14.00-15.00  sec  8.14 MBytes  68.3 Mbits/sec                  
[  5]  15.00-16.00  sec  8.21 MBytes  68.9 Mbits/sec                  
[  5]  16.00-17.00  sec  8.68 MBytes  72.8 Mbits/sec                  
[  5]  17.00-18.00  sec  8.71 MBytes  73.1 Mbits/sec                  
[  5]  18.00-19.00  sec  8.97 MBytes  75.3 Mbits/sec                  
[  5]  19.00-20.00  sec  9.30 MBytes  78.0 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   138 MBytes  57.8 Mbits/sec    3             sender
[  5]   0.00-20.00  sec   134 MBytes  56.3 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 192.168.1.192 port 44002 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  10.4 MBytes  87.6 Mbits/sec    1    662 KBytes       
[  5]   1.00-2.00   sec  10.0 MBytes  83.9 Mbits/sec    0    735 KBytes       
[  5]   2.00-3.00   sec  10.0 MBytes  83.9 Mbits/sec    0    788 KBytes       
[  5]   3.00-4.00   sec  10.0 MBytes  83.9 Mbits/sec    0    824 KBytes       
[  5]   4.00-5.00   sec  10.0 MBytes  83.9 Mbits/sec    0    847 KBytes       
[  5]   5.00-6.00   sec  10.0 MBytes  83.9 Mbits/sec    0    858 KBytes       
[  5]   6.00-7.00   sec  10.0 MBytes  83.9 Mbits/sec    0    863 KBytes       
[  5]   7.00-8.00   sec  8.75 MBytes  73.4 Mbits/sec    0    863 KBytes       
[  5]   8.00-9.00   sec  10.0 MBytes  83.9 Mbits/sec    0    863 KBytes       
[  5]   9.00-10.00  sec  10.0 MBytes  83.9 Mbits/sec    0    867 KBytes       
[  5]  10.00-11.00  sec  10.0 MBytes  83.9 Mbits/sec    0    877 KBytes       
[  5]  11.00-12.00  sec  10.0 MBytes  83.9 Mbits/sec    0    897 KBytes       
[  5]  12.00-13.00  sec  10.0 MBytes  83.9 Mbits/sec    1    792 KBytes       
[  5]  13.00-14.00  sec  10.0 MBytes  83.9 Mbits/sec    1    508 KBytes       
[  5]  14.00-15.00  sec  10.0 MBytes  83.9 Mbits/sec    0    547 KBytes       
[  5]  15.00-16.00  sec  10.0 MBytes  83.9 Mbits/sec    0    573 KBytes       
[  5]  16.00-17.00  sec  10.0 MBytes  83.9 Mbits/sec    0    587 KBytes       
[  5]  17.00-18.00  sec  10.0 MBytes  83.9 Mbits/sec    0    592 KBytes       
[  5]  18.00-19.00  sec  10.0 MBytes  83.9 Mbits/sec    0    592 KBytes       
[  5]  19.00-20.00  sec  10.0 MBytes  83.9 Mbits/sec    0    592 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   199 MBytes  83.5 Mbits/sec    3             sender
[  5]   0.00-20.00  sec   196 MBytes  82.4 Mbits/sec                  receiver

iperf Done.

fq_codel/simple:

IPv6

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 60784 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  8.76 MBytes  73.5 Mbits/sec                  
[  5]   1.00-2.00   sec  9.52 MBytes  79.8 Mbits/sec                  
[  5]   2.00-3.00   sec  9.19 MBytes  77.1 Mbits/sec                  
[  5]   3.00-4.00   sec  9.58 MBytes  80.3 Mbits/sec                  
[  5]   4.00-5.00   sec  9.93 MBytes  83.3 Mbits/sec                  
[  5]   5.00-6.00   sec  10.0 MBytes  84.2 Mbits/sec                  
[  5]   6.00-7.00   sec  10.1 MBytes  84.4 Mbits/sec                  
[  5]   7.00-8.00   sec  10.4 MBytes  87.2 Mbits/sec                  
[  5]   8.00-9.00   sec  10.5 MBytes  88.4 Mbits/sec                  
[  5]   9.00-10.00  sec  10.6 MBytes  88.8 Mbits/sec                  
[  5]  10.00-11.00  sec  10.1 MBytes  84.8 Mbits/sec                  
[  5]  11.00-12.00  sec  9.43 MBytes  79.1 Mbits/sec                  
[  5]  12.00-13.00  sec  10.1 MBytes  84.6 Mbits/sec                  
[  5]  13.00-14.00  sec  10.2 MBytes  85.9 Mbits/sec                  
[  5]  14.00-15.00  sec  10.4 MBytes  87.1 Mbits/sec                  
[  5]  15.00-16.00  sec  10.5 MBytes  88.2 Mbits/sec                  
[  5]  16.00-17.00  sec  9.67 MBytes  81.1 Mbits/sec                  
[  5]  17.00-18.00  sec  9.69 MBytes  81.3 Mbits/sec                  
[  5]  18.00-19.00  sec  10.5 MBytes  88.1 Mbits/sec                  
[  5]  19.00-20.00  sec  10.5 MBytes  88.2 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   202 MBytes  84.7 Mbits/sec   11             sender
[  5]   0.00-20.00  sec   200 MBytes  83.8 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 2a01:79c:cebd:1e2c::a80 port 60788 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  10.3 MBytes  86.1 Mbits/sec    3    402 KBytes       
[  5]   1.00-2.00   sec  10.7 MBytes  90.2 Mbits/sec    0    455 KBytes       
[  5]   2.00-3.00   sec  10.3 MBytes  86.0 Mbits/sec    3    348 KBytes       
[  5]   3.00-4.00   sec  10.1 MBytes  84.5 Mbits/sec    0    376 KBytes       
[  5]   4.00-5.00   sec  10.1 MBytes  84.5 Mbits/sec    0    394 KBytes       
[  5]   5.00-6.00   sec  10.7 MBytes  90.1 Mbits/sec    0    402 KBytes       
[  5]   6.00-7.00   sec  11.4 MBytes  95.8 Mbits/sec    0    408 KBytes       
[  5]   7.00-8.00   sec  10.7 MBytes  90.1 Mbits/sec    0    426 KBytes       
[  5]   8.00-9.00   sec  10.8 MBytes  90.7 Mbits/sec    0    441 KBytes       
[  5]   9.00-10.00  sec  10.7 MBytes  90.2 Mbits/sec    0    457 KBytes       
[  5]  10.00-11.00  sec  11.4 MBytes  95.8 Mbits/sec    0    472 KBytes       
[  5]  11.00-12.00  sec  9.53 MBytes  79.9 Mbits/sec    6    381 KBytes       
[  5]  12.00-13.00  sec  10.7 MBytes  90.1 Mbits/sec    0    421 KBytes       
[  5]  13.00-14.00  sec  10.7 MBytes  90.1 Mbits/sec    0    448 KBytes       
[  5]  14.00-15.00  sec  10.7 MBytes  90.1 Mbits/sec    0    464 KBytes       
[  5]  15.00-16.00  sec  11.1 MBytes  92.7 Mbits/sec    6    333 KBytes       
[  5]  16.00-17.00  sec  9.46 MBytes  79.4 Mbits/sec    0    365 KBytes       
[  5]  17.00-18.00  sec  10.1 MBytes  84.5 Mbits/sec    0    385 KBytes       
[  5]  18.00-19.00  sec  10.7 MBytes  90.1 Mbits/sec    0    395 KBytes       
[  5]  19.00-20.00  sec  10.1 MBytes  84.5 Mbits/sec    0    400 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   210 MBytes  88.3 Mbits/sec   18             sender
[  5]   0.00-20.00  sec   210 MBytes  87.9 Mbits/sec                  receiver

iperf Done.

IPv4

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 192.168.1.192 port 43978 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  8.17 MBytes  68.5 Mbits/sec                  
[  5]   1.00-2.00   sec  8.49 MBytes  71.2 Mbits/sec                  
[  5]   2.00-3.00   sec  9.00 MBytes  75.5 Mbits/sec                  
[  5]   3.00-4.00   sec  9.39 MBytes  78.7 Mbits/sec                  
[  5]   4.00-5.00   sec  9.95 MBytes  83.5 Mbits/sec                  
[  5]   5.00-6.00   sec  9.91 MBytes  83.1 Mbits/sec                  
[  5]   6.00-7.00   sec  8.29 MBytes  69.5 Mbits/sec                  
[  5]   7.00-8.00   sec  5.08 MBytes  42.6 Mbits/sec                  
[  5]   8.00-9.00   sec  5.32 MBytes  44.6 Mbits/sec                  
[  5]   9.00-10.00  sec  5.88 MBytes  49.3 Mbits/sec                  
[  5]  10.00-11.00  sec  6.40 MBytes  53.7 Mbits/sec                  
[  5]  11.00-12.00  sec  7.02 MBytes  58.9 Mbits/sec                  
[  5]  12.00-13.00  sec  7.64 MBytes  64.1 Mbits/sec                  
[  5]  13.00-14.00  sec  8.21 MBytes  68.9 Mbits/sec                  
[  5]  14.00-15.00  sec  8.80 MBytes  73.8 Mbits/sec                  
[  5]  15.00-16.00  sec  9.37 MBytes  78.6 Mbits/sec                  
[  5]  16.00-17.00  sec  9.99 MBytes  83.8 Mbits/sec                  
[  5]  17.00-18.00  sec  8.75 MBytes  73.4 Mbits/sec                  
[  5]  18.00-19.00  sec  8.80 MBytes  73.8 Mbits/sec                  
[  5]  19.00-20.00  sec  9.60 MBytes  80.6 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   167 MBytes  70.0 Mbits/sec    7             sender
[  5]   0.00-20.00  sec   164 MBytes  68.8 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 192.168.1.192 port 43982 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.9 MBytes  99.4 Mbits/sec    2    421 KBytes       
[  5]   1.00-2.00   sec  11.1 MBytes  92.8 Mbits/sec    1    329 KBytes       
[  5]   2.00-3.00   sec  9.94 MBytes  83.4 Mbits/sec    0    366 KBytes       
[  5]   3.00-4.00   sec  11.1 MBytes  93.3 Mbits/sec    0    386 KBytes       
[  5]   4.00-5.00   sec  11.2 MBytes  93.8 Mbits/sec    0    397 KBytes       
[  5]   5.00-6.00   sec  11.1 MBytes  92.8 Mbits/sec    0    416 KBytes       
[  5]   6.00-7.00   sec  11.2 MBytes  93.8 Mbits/sec    0    437 KBytes       
[  5]   7.00-8.00   sec  11.2 MBytes  93.8 Mbits/sec    0    455 KBytes       
[  5]   8.00-9.00   sec  12.2 MBytes   103 Mbits/sec    1    335 KBytes       
[  5]   9.00-10.00  sec  10.1 MBytes  84.4 Mbits/sec    0    392 KBytes       
[  5]  10.00-11.00  sec  11.1 MBytes  92.8 Mbits/sec    0    431 KBytes       
[  5]  11.00-12.00  sec  11.1 MBytes  93.3 Mbits/sec    0    455 KBytes       
[  5]  12.00-13.00  sec  11.2 MBytes  93.8 Mbits/sec    0    468 KBytes       
[  5]  13.00-14.00  sec  11.1 MBytes  93.3 Mbits/sec    8    344 KBytes       
[  5]  14.00-15.00  sec  10.1 MBytes  84.5 Mbits/sec    0    373 KBytes       
[  5]  15.00-16.00  sec  11.1 MBytes  93.3 Mbits/sec    0    390 KBytes       
[  5]  16.00-17.00  sec  11.1 MBytes  93.3 Mbits/sec    0    400 KBytes       
[  5]  17.00-18.00  sec  11.2 MBytes  93.8 Mbits/sec    0    423 KBytes       
[  5]  18.00-19.00  sec  11.1 MBytes  92.8 Mbits/sec    0    443 KBytes       
[  5]  19.00-20.00  sec  11.2 MBytes  93.8 Mbits/sec    0    462 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   221 MBytes  92.8 Mbits/sec   12             sender
[  5]   0.00-20.00  sec   219 MBytes  92.0 Mbits/sec                  receiver

iperf Done.

huaracheguarache · October 1, 2018, 2:03pm

Continued due to character limit:

fq_codel/simplest:

IPv6

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 60860 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  9.15 MBytes  76.8 Mbits/sec                  
[  5]   1.00-2.00   sec  11.0 MBytes  92.0 Mbits/sec                  
[  5]   2.00-3.00   sec  10.6 MBytes  88.8 Mbits/sec                  
[  5]   3.00-4.00   sec  9.21 MBytes  77.2 Mbits/sec                  
[  5]   4.00-5.00   sec  9.73 MBytes  81.6 Mbits/sec                  
[  5]   5.00-6.00   sec  10.2 MBytes  85.3 Mbits/sec                  
[  5]   6.00-7.00   sec  10.2 MBytes  85.9 Mbits/sec                  
[  5]   7.00-8.00   sec  10.3 MBytes  86.3 Mbits/sec                  
[  5]   8.00-9.00   sec  10.3 MBytes  86.8 Mbits/sec                  
[  5]   9.00-10.00  sec  10.5 MBytes  87.9 Mbits/sec                  
[  5]  10.00-11.00  sec  10.5 MBytes  88.4 Mbits/sec                  
[  5]  11.00-12.00  sec  10.5 MBytes  88.2 Mbits/sec                  
[  5]  12.00-13.00  sec  9.66 MBytes  81.1 Mbits/sec                  
[  5]  13.00-14.00  sec  10.1 MBytes  84.8 Mbits/sec                  
[  5]  14.00-15.00  sec  10.3 MBytes  86.2 Mbits/sec                  
[  5]  15.00-16.00  sec  10.4 MBytes  87.4 Mbits/sec                  
[  5]  16.00-17.00  sec  10.4 MBytes  87.5 Mbits/sec                  
[  5]  17.00-18.00  sec  9.41 MBytes  78.9 Mbits/sec                  
[  5]  18.00-19.00  sec  9.82 MBytes  82.4 Mbits/sec                  
[  5]  19.00-20.00  sec  10.0 MBytes  84.0 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   205 MBytes  85.9 Mbits/sec    7             sender
[  5]   0.00-20.00  sec   202 MBytes  84.9 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 2a01:79c:cebd:1e2c::a80 port 60864 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  10.0 MBytes  84.2 Mbits/sec   11    340 KBytes       
[  5]   1.00-2.00   sec  9.59 MBytes  80.4 Mbits/sec    0    373 KBytes       
[  5]   2.00-3.00   sec  10.4 MBytes  87.6 Mbits/sec    0    393 KBytes       
[  5]   3.00-4.00   sec  10.7 MBytes  90.1 Mbits/sec    0    402 KBytes       
[  5]   4.00-5.00   sec  11.4 MBytes  95.8 Mbits/sec    0    407 KBytes       
[  5]   5.00-6.00   sec  10.4 MBytes  87.1 Mbits/sec    0    422 KBytes       
[  5]   6.00-7.00   sec  11.4 MBytes  95.8 Mbits/sec    0    440 KBytes       
[  5]   7.00-8.00   sec  10.4 MBytes  87.6 Mbits/sec    0    455 KBytes       
[  5]   8.00-9.00   sec  11.5 MBytes  96.3 Mbits/sec    3    329 KBytes       
[  5]   9.00-10.00  sec  9.77 MBytes  82.0 Mbits/sec    0    383 KBytes       
[  5]  10.00-11.00  sec  10.4 MBytes  87.1 Mbits/sec    0    422 KBytes       
[  5]  11.00-12.00  sec  10.7 MBytes  90.2 Mbits/sec    0    447 KBytes       
[  5]  12.00-13.00  sec  11.5 MBytes  96.8 Mbits/sec    0    461 KBytes       
[  5]  13.00-14.00  sec  10.4 MBytes  87.6 Mbits/sec    0    468 KBytes       
[  5]  14.00-15.00  sec  9.83 MBytes  82.5 Mbits/sec    3    360 KBytes       
[  5]  15.00-16.00  sec  9.83 MBytes  82.5 Mbits/sec    0    380 KBytes       
[  5]  16.00-17.00  sec  10.7 MBytes  90.1 Mbits/sec    0    392 KBytes       
[  5]  17.00-18.00  sec  10.6 MBytes  88.6 Mbits/sec    0    398 KBytes       
[  5]  18.00-19.00  sec  10.4 MBytes  87.6 Mbits/sec    0    412 KBytes       
[  5]  19.00-20.00  sec  11.4 MBytes  95.8 Mbits/sec    0    429 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   212 MBytes  88.8 Mbits/sec   17             sender
[  5]   0.00-20.00  sec   210 MBytes  88.0 Mbits/sec                  receiver

iperf Done.

IPv4

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 192.168.1.192 port 44052 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  8.79 MBytes  73.7 Mbits/sec                  
[  5]   1.00-2.00   sec  8.53 MBytes  71.6 Mbits/sec                  
[  5]   2.00-3.00   sec  9.07 MBytes  76.1 Mbits/sec                  
[  5]   3.00-4.00   sec  9.50 MBytes  79.7 Mbits/sec                  
[  5]   4.00-5.00   sec  9.98 MBytes  83.7 Mbits/sec                  
[  5]   5.00-6.00   sec  9.39 MBytes  78.8 Mbits/sec                  
[  5]   6.00-7.00   sec  8.65 MBytes  72.6 Mbits/sec                  
[  5]   7.00-8.00   sec  9.62 MBytes  80.7 Mbits/sec                  
[  5]   8.00-9.00   sec  10.2 MBytes  85.9 Mbits/sec                  
[  5]   9.00-10.00  sec  8.98 MBytes  75.3 Mbits/sec                  
[  5]  10.00-11.00  sec  8.19 MBytes  68.7 Mbits/sec                  
[  5]  11.00-12.00  sec  8.68 MBytes  72.8 Mbits/sec                  
[  5]  12.00-13.00  sec  9.09 MBytes  76.2 Mbits/sec                  
[  5]  13.00-14.00  sec  9.63 MBytes  80.8 Mbits/sec                  
[  5]  14.00-15.00  sec  10.2 MBytes  85.4 Mbits/sec                  
[  5]  15.00-16.00  sec  9.13 MBytes  76.5 Mbits/sec                  
[  5]  16.00-17.00  sec  8.91 MBytes  74.8 Mbits/sec                  
[  5]  17.00-18.00  sec  9.82 MBytes  82.4 Mbits/sec                  
[  5]  18.00-19.00  sec  10.4 MBytes  87.1 Mbits/sec                  
[  5]  19.00-20.00  sec  9.41 MBytes  79.0 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   189 MBytes  79.2 Mbits/sec    7             sender
[  5]   0.00-20.00  sec   186 MBytes  78.1 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 192.168.1.192 port 44056 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  11.9 MBytes   100 Mbits/sec    1    424 KBytes       
[  5]   1.00-2.00   sec  11.2 MBytes  93.8 Mbits/sec    0    478 KBytes       
[  5]   2.00-3.00   sec  10.1 MBytes  84.4 Mbits/sec    8    369 KBytes       
[  5]   3.00-4.00   sec  11.2 MBytes  93.8 Mbits/sec    0    392 KBytes       
[  5]   4.00-5.00   sec  11.2 MBytes  93.8 Mbits/sec    0    404 KBytes       
[  5]   5.00-6.00   sec  11.2 MBytes  93.8 Mbits/sec    0    419 KBytes       
[  5]   6.00-7.00   sec  11.2 MBytes  93.8 Mbits/sec    0    440 KBytes       
[  5]   7.00-8.00   sec  11.2 MBytes  93.8 Mbits/sec    0    458 KBytes       
[  5]   8.00-9.00   sec  11.2 MBytes  93.8 Mbits/sec    0    478 KBytes       
[  5]   9.00-10.00  sec  10.1 MBytes  84.4 Mbits/sec    1    387 KBytes       
[  5]  10.00-11.00  sec  11.2 MBytes  93.8 Mbits/sec    0    431 KBytes       
[  5]  11.00-12.00  sec  12.3 MBytes   103 Mbits/sec    0    457 KBytes       
[  5]  12.00-13.00  sec  11.2 MBytes  93.8 Mbits/sec    0    471 KBytes       
[  5]  13.00-14.00  sec  10.1 MBytes  85.0 Mbits/sec    8    349 KBytes       
[  5]  14.00-15.00  sec  10.1 MBytes  84.4 Mbits/sec    0    378 KBytes       
[  5]  15.00-16.00  sec  11.2 MBytes  93.8 Mbits/sec    0    395 KBytes       
[  5]  16.00-17.00  sec  11.2 MBytes  93.8 Mbits/sec    0    404 KBytes       
[  5]  17.00-18.00  sec  11.2 MBytes  93.8 Mbits/sec    0    426 KBytes       
[  5]  18.00-19.00  sec  11.2 MBytes  93.8 Mbits/sec    0    445 KBytes       
[  5]  19.00-20.00  sec  11.2 MBytes  93.8 Mbits/sec    0    465 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   221 MBytes  92.8 Mbits/sec   18             sender
[  5]   0.00-20.00  sec   219 MBytes  92.0 Mbits/sec                  receiver

iperf Done.

fq_codel/simplest_tbf:

IPv6

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 60830 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   319 KBytes  2.61 Mbits/sec                  
[  5]   1.00-2.00   sec   637 KBytes  5.22 Mbits/sec                  
[  5]   2.00-3.00   sec  1.12 MBytes  9.37 Mbits/sec                  
[  5]   3.00-4.00   sec  1.19 MBytes  9.97 Mbits/sec                  
[  5]   4.00-5.00   sec  1.16 MBytes  9.70 Mbits/sec                  
[  5]   5.00-6.00   sec  1.07 MBytes  8.95 Mbits/sec                  
[  5]   6.00-7.00   sec  1.46 MBytes  12.2 Mbits/sec                  
[  5]   7.00-8.00   sec  1.29 MBytes  10.8 Mbits/sec                  
[  5]   8.00-9.00   sec  1020 KBytes  8.36 Mbits/sec                  
[  5]   9.00-10.00  sec  1.42 MBytes  11.9 Mbits/sec                  
[  5]  10.00-11.00  sec  1.40 MBytes  11.7 Mbits/sec                  
[  5]  11.00-12.00  sec  1.85 MBytes  15.5 Mbits/sec                  
[  5]  12.00-13.00  sec  2.02 MBytes  17.0 Mbits/sec                  
[  5]  13.00-14.00  sec  1.91 MBytes  16.0 Mbits/sec                  
[  5]  14.00-15.00  sec  1.02 MBytes  8.56 Mbits/sec                  
[  5]  15.00-16.00  sec  1.17 MBytes  9.82 Mbits/sec                  
[  5]  16.00-17.00  sec   822 KBytes  6.74 Mbits/sec                  
[  5]  17.00-18.00  sec   676 KBytes  5.54 Mbits/sec                  
[  5]  18.00-19.00  sec   455 KBytes  3.73 Mbits/sec                  
[  5]  19.00-20.00  sec   559 KBytes  4.58 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec  22.7 MBytes  9.51 Mbits/sec  150             sender
[  5]   0.00-20.00  sec  22.4 MBytes  9.42 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 2a01:79c:cebd:1e2c::a80 port 60836 connected to 2001:860:deff:1000::2 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  12.0 MBytes   100 Mbits/sec   13    368 KBytes       
[  5]   1.00-2.00   sec  10.0 MBytes  83.9 Mbits/sec    0    405 KBytes       
[  5]   2.00-3.00   sec  11.2 MBytes  94.4 Mbits/sec    0    426 KBytes       
[  5]   3.00-4.00   sec  10.0 MBytes  83.9 Mbits/sec    0    439 KBytes       
[  5]   4.00-5.00   sec  11.2 MBytes  94.4 Mbits/sec    0    444 KBytes       
[  5]   5.00-6.00   sec  10.0 MBytes  83.9 Mbits/sec    0    447 KBytes       
[  5]   6.00-7.00   sec  11.2 MBytes  94.4 Mbits/sec    5    328 KBytes       
[  5]   7.00-8.00   sec  8.75 MBytes  73.4 Mbits/sec    0    380 KBytes       
[  5]   8.00-9.00   sec  11.2 MBytes  94.4 Mbits/sec    0    416 KBytes       
[  5]   9.00-10.00  sec  11.2 MBytes  94.4 Mbits/sec    0    440 KBytes       
[  5]  10.00-11.00  sec  10.0 MBytes  83.9 Mbits/sec    0    453 KBytes       
[  5]  11.00-12.00  sec  11.2 MBytes  94.4 Mbits/sec    0    459 KBytes       
[  5]  12.00-13.00  sec  10.0 MBytes  83.9 Mbits/sec    0    460 KBytes       
[  5]  13.00-14.00  sec  11.2 MBytes  94.4 Mbits/sec    1    324 KBytes       
[  5]  14.00-15.00  sec  8.75 MBytes  73.4 Mbits/sec    0    378 KBytes       
[  5]  15.00-16.00  sec  11.2 MBytes  94.4 Mbits/sec    0    414 KBytes       
[  5]  16.00-17.00  sec  10.0 MBytes  83.9 Mbits/sec    0    439 KBytes       
[  5]  17.00-18.00  sec  11.2 MBytes  94.4 Mbits/sec    0    453 KBytes       
[  5]  18.00-19.00  sec  10.0 MBytes  83.9 Mbits/sec    0    459 KBytes       
[  5]  19.00-20.00  sec  10.0 MBytes  83.9 Mbits/sec    1    349 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   211 MBytes  88.4 Mbits/sec   20             sender
[  5]   0.00-20.00  sec   208 MBytes  87.1 Mbits/sec                  receiver

iperf Done.

IPv4

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -R -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 192.168.1.192 port 44024 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  9.75 MBytes  81.7 Mbits/sec                  
[  5]   1.00-2.00   sec  11.3 MBytes  94.4 Mbits/sec                  
[  5]   2.00-3.00   sec  9.99 MBytes  83.8 Mbits/sec                  
[  5]   3.00-4.00   sec  10.6 MBytes  88.8 Mbits/sec                  
[  5]   4.00-5.00   sec  11.0 MBytes  92.2 Mbits/sec                  
[  5]   5.00-6.00   sec  11.2 MBytes  93.8 Mbits/sec                  
[  5]   6.00-7.00   sec  11.3 MBytes  95.0 Mbits/sec                  
[  5]   7.00-8.00   sec  11.3 MBytes  95.1 Mbits/sec                  
[  5]   8.00-9.00   sec  11.3 MBytes  95.0 Mbits/sec                  
[  5]   9.00-10.00  sec  10.2 MBytes  85.7 Mbits/sec                  
[  5]  10.00-11.00  sec  11.1 MBytes  92.9 Mbits/sec                  
[  5]  11.00-12.00  sec  11.3 MBytes  95.0 Mbits/sec                  
[  5]  12.00-13.00  sec  11.3 MBytes  94.8 Mbits/sec                  
[  5]  13.00-14.00  sec  11.2 MBytes  94.1 Mbits/sec                  
[  5]  14.00-15.00  sec  9.91 MBytes  83.1 Mbits/sec                  
[  5]  15.00-16.00  sec  10.6 MBytes  89.2 Mbits/sec                  
[  5]  16.00-17.00  sec  11.0 MBytes  92.1 Mbits/sec                  
[  5]  17.00-18.00  sec  11.2 MBytes  94.0 Mbits/sec                  
[  5]  18.00-19.00  sec  11.3 MBytes  95.1 Mbits/sec                  
[  5]  19.00-20.00  sec  11.4 MBytes  95.3 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   221 MBytes  92.5 Mbits/sec    5             sender
[  5]   0.00-20.00  sec   218 MBytes  91.6 Mbits/sec                  receiver

iperf Done.

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -4 -p 5206 -t 20
Connecting to host bouygues.iperf.fr, port 5206
[  5] local 192.168.1.192 port 44028 connected to 89.84.1.222 port 5206
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  12.3 MBytes   103 Mbits/sec    1    417 KBytes       
[  5]   1.00-2.00   sec  10.9 MBytes  91.7 Mbits/sec    0    471 KBytes       
[  5]   2.00-3.00   sec  9.76 MBytes  81.8 Mbits/sec    1    365 KBytes       
[  5]   3.00-4.00   sec  10.9 MBytes  91.7 Mbits/sec    0    389 KBytes       
[  5]   4.00-5.00   sec  10.8 MBytes  90.7 Mbits/sec    0    402 KBytes       
[  5]   5.00-6.00   sec  12.1 MBytes   101 Mbits/sec    0    416 KBytes       
[  5]   6.00-7.00   sec  10.8 MBytes  90.2 Mbits/sec    0    436 KBytes       
[  5]   7.00-8.00   sec  12.0 MBytes   101 Mbits/sec    0    455 KBytes       
[  5]   8.00-9.00   sec  10.7 MBytes  89.7 Mbits/sec    1    334 KBytes       
[  5]   9.00-10.00  sec  9.82 MBytes  82.4 Mbits/sec    0    390 KBytes       
[  5]  10.00-11.00  sec  11.9 MBytes  99.6 Mbits/sec    0    430 KBytes       
[  5]  11.00-12.00  sec  10.8 MBytes  90.2 Mbits/sec    0    454 KBytes       
[  5]  12.00-13.00  sec  11.9 MBytes  99.6 Mbits/sec    0    467 KBytes       
[  5]  13.00-14.00  sec  10.8 MBytes  90.2 Mbits/sec    1    346 KBytes       
[  5]  14.00-15.00  sec  9.82 MBytes  82.4 Mbits/sec    0    375 KBytes       
[  5]  15.00-16.00  sec  10.9 MBytes  91.8 Mbits/sec    0    390 KBytes       
[  5]  16.00-17.00  sec  10.8 MBytes  90.7 Mbits/sec    0    403 KBytes       
[  5]  17.00-18.00  sec  10.9 MBytes  91.7 Mbits/sec    0    423 KBytes       
[  5]  18.00-19.00  sec  11.9 MBytes   100 Mbits/sec    0    444 KBytes       
[  5]  19.00-20.00  sec  10.8 MBytes  90.7 Mbits/sec    0    462 KBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   221 MBytes  92.5 Mbits/sec    4             sender
[  5]   0.00-20.00  sec   218 MBytes  91.6 Mbits/sec                  receiver

iperf Done.

@moeller0 do you know why this is happening?

moeller0 · October 1, 2018, 3:01pm

Mmmh, try to disable all TSO. GSO, GRO offloading (by using ethtool), there was a bug with cake's super-packet splitter (see https://github.com/tohojo/sqm-scripts/issues/72) and double encapsulated packets like in 6rd. I had thought that the fix made into the recent snap shots, but apparently either that or the fix is not yet complete.

@tohojo, does any of this ring a bell?

tohojo · October 1, 2018, 3:21pm

Yup, definitely sounds like the GSO splitting bug; turning off GRO on the wan interface should fix it, until the upstream fix makes it into openwrt

huaracheguarache · October 1, 2018, 3:38pm

Ok, I tried disabling TSO, GSO and GRO on the wan interface, but it didn't have any effect:

root@telia:~# ethtool -K eth0.102 gso off gro off tso off

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5207 -t 20
Connecting to host bouygues.iperf.fr, port 5207
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::5bb port 49382 connected to 2001:860:deff:1000::2 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  97.9 KBytes   802 Kbits/sec                  
[  5]   1.00-2.00   sec  63.7 KBytes   522 Kbits/sec                  
[  5]   2.00-3.00   sec  82.6 KBytes   676 Kbits/sec                  
[  5]   3.00-4.00   sec  57.8 KBytes   474 Kbits/sec                  
[  5]   4.00-5.00   sec  64.9 KBytes   532 Kbits/sec                  
[  5]   5.00-6.00   sec  60.2 KBytes   493 Kbits/sec                  
[  5]   6.00-7.00   sec  81.4 KBytes   667 Kbits/sec                  
[  5]   7.00-8.00   sec  77.9 KBytes   638 Kbits/sec                  
[  5]   8.00-9.00   sec  35.4 KBytes   290 Kbits/sec                  
[  5]   9.00-10.00  sec  29.5 KBytes   242 Kbits/sec                  
[  5]  10.00-11.00  sec  41.3 KBytes   338 Kbits/sec                  
[  5]  11.00-12.00  sec  73.1 KBytes   599 Kbits/sec                  
[  5]  12.00-13.00  sec  82.6 KBytes   676 Kbits/sec                  
[  5]  13.00-14.00  sec  82.6 KBytes   676 Kbits/sec                  
[  5]  14.00-15.00  sec  47.2 KBytes   387 Kbits/sec                  
[  5]  15.00-16.00  sec  66.1 KBytes   541 Kbits/sec                  
[  5]  16.00-17.00  sec  54.3 KBytes   444 Kbits/sec                  
[  5]  17.00-18.00  sec  94.4 KBytes   773 Kbits/sec                  
[  5]  18.00-19.00  sec  60.2 KBytes   493 Kbits/sec                  
[  5]  19.00-20.00  sec  49.5 KBytes   406 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec  1.30 MBytes   546 Kbits/sec  257             sender
[  5]   0.00-20.00  sec  1.27 MBytes   533 Kbits/sec                  receiver

iperf Done.

I also tried to disable only GRO, but I got the same results:

root@telia:~# ethtool -K eth0.102 gro off

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5207 -t 20
Connecting to host bouygues.iperf.fr, port 5207
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::5bb port 49390 connected to 2001:860:deff:1000::2 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  70.8 KBytes   580 Kbits/sec                  
[  5]   1.00-2.00   sec  51.9 KBytes   425 Kbits/sec                  
[  5]   2.00-3.00   sec  42.5 KBytes   348 Kbits/sec                  
[  5]   3.00-4.00   sec  57.8 KBytes   474 Kbits/sec                  
[  5]   4.00-5.00   sec  60.2 KBytes   493 Kbits/sec                  
[  5]   5.00-6.00   sec  64.9 KBytes   532 Kbits/sec                  
[  5]   6.00-7.00   sec  54.3 KBytes   444 Kbits/sec                  
[  5]   7.00-8.00   sec  50.7 KBytes   416 Kbits/sec                  
[  5]   8.00-9.00   sec  40.1 KBytes   329 Kbits/sec                  
[  5]   9.00-10.00  sec  44.8 KBytes   367 Kbits/sec                  
[  5]  10.00-11.00  sec  68.4 KBytes   561 Kbits/sec                  
[  5]  11.00-12.00  sec  66.1 KBytes   541 Kbits/sec                  
[  5]  12.00-13.00  sec  50.7 KBytes   416 Kbits/sec                  
[  5]  13.00-14.00  sec  57.8 KBytes   473 Kbits/sec                  
[  5]  14.00-15.00  sec  56.6 KBytes   464 Kbits/sec                  
[  5]  15.00-16.00  sec  73.1 KBytes   599 Kbits/sec                  
[  5]  16.00-17.00  sec  49.5 KBytes   406 Kbits/sec                  
[  5]  17.00-18.00  sec  62.5 KBytes   512 Kbits/sec                  
[  5]  18.00-19.00  sec  67.2 KBytes   551 Kbits/sec                  
[  5]  19.00-20.00  sec  51.9 KBytes   425 Kbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec  1.17 MBytes   490 Kbits/sec  252             sender
[  5]   0.00-20.00  sec  1.12 MBytes   468 Kbits/sec                  receiver

iperf Done.

But adding the 'no-split-gso' parameter to cake in the advanced settings worked:

root@telia:~# cat /etc/config/sqm 

config queue 'eth1'
	option debug_logging '0'
	option verbosity '5'
	option interface 'eth0.102'
	option linklayer 'none'
	option download '100000'
	option upload '100000'
	option enabled '1'
	option qdisc 'cake'
	option script 'layer_cake.qos'
	option qdisc_advanced '1'
	option squash_dscp '1'
	option squash_ingress '1'
	option ingress_ecn 'ECN'
	option egress_ecn 'NOECN'
	option qdisc_really_really_advanced '1'
	option iqdisc_opts 'no-split-gso'
	option eqdisc_opts 'no-split-gso'


[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5207 -t 20
Connecting to host bouygues.iperf.fr, port 5207
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::5bb port 49404 connected to 2001:860:deff:1000::2 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   685 KBytes  5.61 Mbits/sec                  
[  5]   1.00-2.00   sec  2.47 MBytes  20.7 Mbits/sec                  
[  5]   2.00-3.00   sec  7.75 MBytes  65.0 Mbits/sec                  
[  5]   3.00-4.00   sec  9.02 MBytes  75.7 Mbits/sec                  
[  5]   4.00-5.00   sec  8.81 MBytes  73.9 Mbits/sec                  
[  5]   5.00-6.00   sec  8.99 MBytes  75.4 Mbits/sec                  
[  5]   6.00-7.00   sec  9.16 MBytes  76.8 Mbits/sec                  
[  5]   7.00-8.00   sec  9.31 MBytes  78.1 Mbits/sec                  
[  5]   8.00-9.00   sec  9.13 MBytes  76.6 Mbits/sec                  
[  5]   9.00-10.00  sec  8.26 MBytes  69.3 Mbits/sec                  
[  5]  10.00-11.00  sec  8.56 MBytes  71.8 Mbits/sec                  
[  5]  11.00-12.00  sec  9.08 MBytes  76.1 Mbits/sec                  
[  5]  12.00-13.00  sec  9.23 MBytes  77.4 Mbits/sec                  
[  5]  13.00-14.00  sec  9.44 MBytes  79.2 Mbits/sec                  
[  5]  14.00-15.00  sec  9.46 MBytes  79.4 Mbits/sec                  
[  5]  15.00-16.00  sec  9.58 MBytes  80.3 Mbits/sec                  
[  5]  16.00-17.00  sec  9.54 MBytes  80.0 Mbits/sec                  
[  5]  17.00-18.00  sec  9.63 MBytes  80.8 Mbits/sec                  
[  5]  18.00-19.00  sec  9.77 MBytes  82.0 Mbits/sec                  
[  5]  19.00-20.00  sec  9.93 MBytes  83.3 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   170 MBytes  71.2 Mbits/sec    5             sender
[  5]   0.00-20.00  sec   168 MBytes  70.4 Mbits/sec                  receiver

iperf Done.

huaracheguarache · October 1, 2018, 3:46pm

I thought the issue had been fixed in this workaround: https://git.openwrt.org/?p=openwrt/openwrt.git;a=commit;h=66fd41ba79356b1b776ba29dd8191039abc23061

moeller0 · October 1, 2018, 4:29pm

I wonder whether disabling all the offloads on eth0 instead might work better?

huaracheguarache · October 1, 2018, 8:10pm

Yes, that worked!

I'm unable to turn off tso on eth0 as can been seen in the output of ethtool -k eth0 :

tcp-segmentation-offload: off
	tx-tcp-segmentation: off [fixed]
	tx-tcp-ecn-segmentation: off [fixed]
	tx-tcp-mangleid-segmentation: off [fixed]
	tx-tcp6-segmentation: off [fixed]

But turning off gso and gro works:

root@telia:~# ethtool -K eth0 gso off gro off tso off
Cannot change tcp-segmentation-offload

root@telia:~# ethtool -k eth0

generic-segmentation-offload: off
generic-receive-offload: off

And speeds are normal:

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5207 -t 20
Connecting to host bouygues.iperf.fr, port 5207
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c::a80 port 44030 connected to 2001:860:deff:1000::2 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  3.07 MBytes  25.8 Mbits/sec                  
[  5]   1.00-2.00   sec  8.15 MBytes  68.4 Mbits/sec                  
[  5]   2.00-3.00   sec  10.2 MBytes  85.2 Mbits/sec                  
[  5]   3.00-4.00   sec  10.5 MBytes  87.8 Mbits/sec                  
[  5]   4.00-5.00   sec  9.69 MBytes  81.3 Mbits/sec                  
[  5]   5.00-6.00   sec  9.32 MBytes  78.2 Mbits/sec                  
[  5]   6.00-7.00   sec  9.79 MBytes  82.1 Mbits/sec                  
[  5]   7.00-8.00   sec  9.86 MBytes  82.7 Mbits/sec                  
[  5]   8.00-9.00   sec  10.2 MBytes  85.8 Mbits/sec                  
[  5]   9.00-10.00  sec  10.0 MBytes  84.0 Mbits/sec                  
[  5]  10.00-11.00  sec  10.2 MBytes  85.8 Mbits/sec                  
[  5]  11.00-12.00  sec  10.1 MBytes  84.9 Mbits/sec                  
[  5]  12.00-13.00  sec  10.5 MBytes  88.1 Mbits/sec                  
[  5]  13.00-14.00  sec  10.6 MBytes  89.0 Mbits/sec                  
[  5]  14.00-15.00  sec  10.7 MBytes  90.1 Mbits/sec                  
[  5]  15.00-16.00  sec  10.4 MBytes  87.1 Mbits/sec                  
[  5]  16.00-17.00  sec  9.48 MBytes  79.6 Mbits/sec                  
[  5]  17.00-18.00  sec  10.1 MBytes  84.6 Mbits/sec                  
[  5]  18.00-19.00  sec  10.8 MBytes  90.4 Mbits/sec                  
[  5]  19.00-20.00  sec  10.8 MBytes  90.4 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   196 MBytes  82.4 Mbits/sec    4             sender
[  5]   0.00-20.00  sec   194 MBytes  81.6 Mbits/sec                  receiver

iperf Done.

huaracheguarache · October 2, 2018, 8:44am

The upstream fix will arrive in kernel 4.14.73 and 4.9.130. I tested @ldir's patches bumping the kernel to 4.14.73 and disabling cake's workaround:

https://git.openwrt.org/?p=openwrt/staging/ldir.git;a=commit;h=b92bd7c6a2f98e065f982742cee486ab638f5fc6

https://git.openwrt.org/?p=openwrt/staging/ldir.git;a=commit;h=10310ecc3decdf7078b04c5f28f84b672ef6cdd2

It fixed the issue:

[michael@linko ~]$ iperf3 -c bouygues.iperf.fr -6 -R -p 5207 -t 20
Connecting to host bouygues.iperf.fr, port 5207
Reverse mode, remote host bouygues.iperf.fr is sending
[  5] local 2a01:79c:cebd:1e2c:a60f:98e:f951:7015 port 46864 connected to 2001:860:deff:1000::2 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.41 MBytes  20.2 Mbits/sec                  
[  5]   1.00-2.00   sec  3.02 MBytes  25.3 Mbits/sec                  
[  5]   2.00-3.00   sec  3.25 MBytes  27.2 Mbits/sec                  
[  5]   3.00-4.00   sec  3.56 MBytes  29.9 Mbits/sec                  
[  5]   4.00-5.00   sec  3.87 MBytes  32.5 Mbits/sec                  
[  5]   5.00-6.00   sec  4.17 MBytes  35.0 Mbits/sec                  
[  5]   6.00-7.00   sec  4.48 MBytes  37.6 Mbits/sec                  
[  5]   7.00-8.00   sec  4.86 MBytes  40.8 Mbits/sec                  
[  5]   8.00-9.00   sec  5.20 MBytes  43.6 Mbits/sec                  
[  5]   9.00-10.00  sec  5.54 MBytes  46.5 Mbits/sec                  
[  5]  10.00-11.00  sec  5.80 MBytes  48.6 Mbits/sec                  
[  5]  11.00-12.00  sec  6.55 MBytes  55.0 Mbits/sec                  
[  5]  12.00-13.00  sec  8.02 MBytes  67.3 Mbits/sec                  
[  5]  13.00-14.00  sec  10.0 MBytes  84.2 Mbits/sec                  
[  5]  14.00-15.00  sec  9.90 MBytes  83.1 Mbits/sec                  
[  5]  15.00-16.00  sec  9.99 MBytes  83.8 Mbits/sec                  
[  5]  16.00-17.00  sec  10.8 MBytes  90.8 Mbits/sec                  
[  5]  17.00-18.00  sec  10.8 MBytes  90.8 Mbits/sec                  
[  5]  18.00-19.00  sec  9.57 MBytes  80.3 Mbits/sec                  
[  5]  19.00-20.00  sec  9.25 MBytes  77.5 Mbits/sec                  
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-20.00  sec   133 MBytes  55.7 Mbits/sec    3             sender
[  5]   0.00-20.00  sec   131 MBytes  55.0 Mbits/sec                  receiver

iperf Done.

I find it strange that the workaround implemented in cake didn't work for me.

ldir · October 2, 2018, 9:30am

If I were really clever I'd work out the kernel version includes in such a way that the conditional gso fix could go into the cake repo and then openwrt wouldn't have to carry a patch.

Thanks for testing what I do have at the moment