Speedtest: new package to measure network performance


#41

It shows this way in ipq806x R7800:

root@router1:/tmp# ./speedtest.sh
2018-12-18 21:56:55 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf.bufferbloat.net (IPv4) while pinging gstatic.com.
Download and upload sessions are sequential, each with 5 simultaneous streams.
.............................................................
 Download:  80.41 Mbps
  Latency: [in msec, 61 pings, 0.00% packet loss]
      Min:  11.478
    10pct:  11.572
   Median:  11.838
      Avg:  11.901
    90pct:  12.298
      Max:  12.686
 CPU Load: [in % busy (avg +/- std dev), 57 samples]
     cpu0:  35.3% +/- 17.8%  @ 1283 MHz
     cpu1:  35.9% +/- 19.3%  @ 1260 MHz
 Overhead: [in % used of total CPU available]
  netperf: 21.0%
.............................................................
   Upload:   9.19 Mbps
  Latency: [in msec, 62 pings, 0.00% packet loss]
      Min:  11.489
    10pct:  11.617
   Median:  12.308
      Avg:  12.382
    90pct:  13.118
      Max:  14.770
 CPU Load: [in % busy (avg +/- std dev), 58 samples]
     cpu0:  16.1% +/- 13.4%  @  967 MHz
     cpu1:  14.9% +/- 14.9%  @  866 MHz
 Overhead: [in % used of total CPU available]
  netperf:  2.5%
root@router1:/tmp# ./speedtest.sh -c
2018-12-18 21:59:09 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf.bufferbloat.net (IPv4) while pinging gstatic.com.
Download and upload sessions are concurrent, each with 5 simultaneous streams.
.............................................................
 Download:  80.40 Mbps
   Upload:   8.18 Mbps
  Latency: [in msec, 61 pings, 0.00% packet loss]
      Min:  11.409
    10pct:  11.758
   Median:  12.505
      Avg:  12.708
    90pct:  13.526
      Max:  17.203
 CPU Load: [in % busy (avg +/- std dev), 57 samples]
     cpu0:  38.4% +/- 19.2%  @ 1294 MHz
     cpu1:  41.2% +/- 15.6%  @ 1297 MHz
 Overhead: [in % used of total CPU available]
  netperf: 20.8%

#42

Hi! I´ve done some very basic benchmarks to figure out if an Asus AC68U (ARM dual core @ 800MHz) has enough CPU to manage SQM in a 100 / 100 mbit connection. The aim is to use VLAN in the router as well, but I forgot to also try it out during the tests.

SQM settings in OpenWRT (18.06.1) was set to 100 000 for both upload and download. Queue Discipline was set to Cake and Piece of Cake.

The test setup consists of a local netperf server in my LAN which the Asus AC68U router was connected to from its WAN port. Not sure I get realistic results from limiting the throughput using SQM parameters in the router compared applying the limitations further upstream.

Finally I installed the "speetest package" from: https://github.com/guidosarducci/packages/tree/master-add-speedtest/net/speedtest/files

I then ran this command using different CPU frequencies : speedtest.sh -H -p 1.1.1.1 --concurrent

Below are test results:

CPU 400 Mhz
Memory: 800 Mhz

Download: 93.20 Mbps
Upload: 93.96 Mbps
Latency: [in msec, 60 pings, 0.00% packet loss]
Min: 8.280
10pct: 8.966
Median: 10.038
Avg: 10.225
90pct: 11.475
Max: 16.441
CPU Load: [in % busy (avg +/- std dev), 55 samples]
cpu0: 96.2% +/- 2.0%
cpu1: 99.7% +/- 0.2%
Overhead: [in % used of total CPU available]
netperf: 69.5%

CPU 1200 Mhz
Memory: 800 Mhz

Download:  93.02 Mbps
   Upload:  93.27 Mbps
  Latency: [in msec, 59 pings, 0.00% packet loss]
      Min:   8.360
    10pct:   8.597
   Median:   9.043
      Avg:   9.281
    90pct:  10.396
      Max:  11.201
 CPU Load: [in % busy (avg +/- std dev), 56 samples]
     cpu0:  72.4% +/-  4.1%
     cpu1:  96.3% +/-  2.0%
 Overhead: [in % used of total CPU available]
  netperf: 35.2%

CPU 1400 Mhz
Memory: 800 Mhz

Download:  91.16 Mbps
   Upload:  93.19 Mbps
  Latency: [in msec, 60 pings, 0.00% packet loss]
      Min:   8.197
    10pct:   8.349
   Median:   8.856
      Avg:   9.079
    90pct:   9.600
      Max:  16.938
 CPU Load: [in % busy (avg +/- std dev), 57 samples]
     cpu0:  66.2% +/-  6.5%
     cpu1:  88.3% +/-  3.1%
 Overhead: [in % used of total CPU available]
  netperf: 37.3%

Is the setup of the test correct? If so, what conclusions can I draw from the results?

Thanks! Erik


#43

I would say the test show that shaping 100/100 works on your router even clocked down to 400 MHz, assuming the frequency stayed constant during the test (standard deviations would be nice to have for frequency as already implemented for the load). But you probably should not run heavy applications like netperf when running at 400 MHz (with load in the high nineties you basically have no reserve cycles left for say the wifi driver).


#44

Thanks for the feedback moeller0! 1200Mhz CPU + 800Mhz memory seems to be a good balance between avoiding additional cooling yet staying below 90% for most cores.

Wifi is not a problem because I didn´t plan to use it.

Targeted a netperf server outside of my LAN, adding two samples to the post.

Local netperf server netperf-eu.bufferbloat.net sample #1 netperf-eu.bufferbloat.net sample #2

Download: 93.02 Mbps
Upload: 93.27 Mbps
Latency: [in msec, 59 pings, 0.00% packet loss]
Min: 8.360
10pct: 8.597
Median: 9.043
Avg: 9.281
90pct: 10.396
Max: 11.201
CPU Load: [in % busy (avg +/- std dev), 56 samples]
cpu0: 72.4% +/- 4.1%
cpu1: 96.3% +/- 2.0%
Overhead: [in % used of total CPU available]
netperf: 35.2%

Download: 86.95 Mbps
Upload: 93.61 Mbps
Latency: [in msec, 60 pings, 0.00% packet loss]
Min: 8.264
10pct: 8.405
Median: 8.834
Avg: 8.952
90pct: 9.562
Max: 10.668
CPU Load: [in % busy (avg +/- std dev), 56 samples]
cpu0: 80.0% +/- 5.4%
cpu1: 86.4% +/- 5.6%
Overhead: [in % used of total CPU available]
netperf: 49.0%

Download: 89.26 Mbps
Upload: 93.50 Mbps
Latency: [in msec, 60 pings, 0.00% packet loss]
Min: 8.431
10pct: 8.602
Median: 9.231
Avg: 9.554
90pct: 10.538
Max: 14.159
CPU Load: [in % busy (avg +/- std dev), 56 samples]
cpu0: 82.2% +/- 3.9%
cpu1: 89.8% +/- 4.3%
Overhead: [in % used of total CPU available]
netperf: 51.6%

Using a local netperf server the CPU load is less balanced beteen both cores, and netperf used significantly less CPU (35% vs 50%).

The results still suggests its possible to get a rough indication of where the CPU limit lies for a givven CPU.

EDIT #1: fixed typo


#45

The AC68U runs pretty cool and I see no realistic difference in temperature in regular use between different frequencies. Did you notice overheating at 1400 ? I played around with these too, and my temps always remain around 61 C for regular use. Also, I can see sqm working fine for my 150/10 connection with default clocks. Bumping it up to 1400 MHz hasn't really shown any improvement in anything other than synthetic compute tests.


#46

Reading at snb forums some people mentioned issues with high temperatures but I was surprised to find the temperature of my AC68U rev A2 staying around 65 degrees.

Running it at 1400 Mhz I noticed barely any change in temperature at all but if I remember correctly, the router got slightly more unstable getting a reboot once in a while. Yet feeling with my hand the router didn´t get any warmer. Tried adding a 120mm fan to cool the router but I noticed no big change in temperature but the router stabilized. So apparently some component got cooler, although the router never felt hot nor reported high temps at all, regardless of using fan or not. Makes me question the accuracy of using /sys/class/thermal/thermal_zone0/temp to get temps - or if the stability was impacted by something else.

Then I totally lost my mind and went directly for 1600 Mhz. The router still booted but then got into a loop after some time. Managed to restore settings using Asus Recovery Utility after a period of cool-down.

I tend to spend way too much time to reach marginal benefits which probably isn´t very noticeable in real world usage. All these rabbits holes are just too tempting..! :slight_smile:


#47

Yes, I don't think any freq more than 1400 is supported. I tried increasing that too, and had a similar experience. Another factor is that since there is no wifi with lede, at least those parts are not even powered on presumably. So that also helps keep the overall temp in check. The other 68U that I'm using as an AP with merlin fork shows a higher temp of 72C for the cpu with 1200 MHz. Also, this is actually a pretty good cheap setup for < $100 with two AC68Us. I imagine it will handle sqm fine up to 300 Mbps.


#48

Almost, one issue the grand average CPU load hides is that the CPU might be cyclically overloaded, say at 50% load over a 10 second test the CPU might be running at 100% for 5 seconds and zero for the other 5 seconds; in that case expect bad shaping performance. Now, having the average load is really much better than having no measure of load at all, but it is not exactly temporally high-resolved enough to show "micro-stalls".