Speedtest: new package to measure network performance

I am using MWAN3 and I wonder if there is a way to run this speedtest script for each wan interface.
I found there is -L option in netperf to specify local IP.
However looking at the results I can see that it's not really using the specified local interface. It uses primary wan link irrespective of what I provide in -L option.
Is a higher layer rule (metric, default route etc) over-riding this option?
Any direction appreciated.

The script doesn't define the route packets take to get to the netperf servers; that's something you need to define as part of your system configuration. It's no different than running wget and expecting it to control routing; it's not meant to.

You haven't really explained what you're trying to accomplish or how you've configured things. IIRC, MWAN3 (or another policy routing application like vpn-policy-routing) should let you configure routing based on destination. So, for example, forcing all traffic destined for netperf-east.bufferbloat.net to go through wan0 rather wan1 should be possible.

Since you're using MWAN3, why not configure an appropriate rule there? Or even simpler, how about shutting down one wan interface to force the other to be used?

That leads to network disruption for a minute while mwan3 failover happens, which I want to avoid.
My config is simple, 2 WANs, using mwan3 as fail-over. One active, other idle/standby.
Thanks for the pointers, I will check if mwan3 has interface selection option based on destination.
Also, I was hoping the netperf -L option would work, which didn't, probably mwan3 interfered here.
Wasn't expecting the speedtest script to do much on this.

==========
update:

I was able to set user rule in mwan3 and do "mwan3 restart" to take effect.
This way I could select wan links and run speedtest for that link.

config rule 'speedtest'
list comment 'to speedtest server'
option src_ip '0.0.0.0/0'
option dest_ip 'xx.xx.xx.xx/32'
option use_policy 'wan_only'

==============
update:
Even better, added a new rule for speedtest server using "ip rule add ..."
Doesn't require mwan restart.. so no network disruption at all.

Hi all, some news about the package:

I've folded in some final changes I've been trialing:

  • Updated the package name to clarify the back-end, highlight the ability to use any netperf server, and distinguish from other speedtest variants. Users should use the new speedtest-netperf_1.0.0-1_all.ipk package after uninstalling the old one. The new package will remind you of any conflict with the old one during installation.
  • Improved consistency and clarity of results and units.
  • Check for presence of netperf if running the unpackaged script on a Linux server.

Finally, the package has been merged into the openwrt/packages repo. It can now be installed on any snapshot build using opkg directly, or downloaded from the OpenWRT download site (note that the package is architecture independent).

I'll update the top post accordingly. Thanks to everyone for feedback and testing, and please continue using this topic for discussion!

6 Likes

People should keep in mind the impacts that QoS can have on CWND sizing. It takes some really big windows (in the thousands) to flood a gigabit pipe using a handful of streams.
If running QoS in the router, it tends to force the TCP stacks to back off on their CWND sizing (in the double digits or low triple digits) to achieve low-latencies. So one needs to use a LOT of concurrent streams.
Testing from browsers on clients, it could take 32 streams (or more) to get enough data on the line.

1 Like

FYI this package works for 18.0.6.2 stable as well.

1 Like

Hmm I'm on OpenWrt 18.06.2 r7676-cddd7b4c77 but don't see the option to install speedtest in the software tab.

The package has not been officially backported to the stable 18.06 branch, so it only exists in the master development branch.

You can install it manually from the .ipk file (or just copy the script).

To install directly from a snapshot of the master branch:
(note the package is arch independent)

# cd /tmp
# uclient-fetch https://downloads.openwrt.org/snapshots/packages/mips_24kc/packages/speedtest-netperf_1.0.0-1_all.ipk
# opkg install speedtest-netperf_1.0.0-1_all.ipk
2 Likes

same situation, the AC68U runs pretty cool and I see no realistic difference in temperature in regular use between different frequencies.

A wee update: the package has now been officially backported to the stable 18.06 branch, which should provide easier access for most folks.

4 Likes

Thanks for the package. I tried to install it on my Archer C7 with OpenWrt 18.06.1 r7258-5eb055306f and ran into a problem.

I updated all the package lists, and then went to "S" to find speedtest-netperf When I clicked Install, I got the message below. What other info could I collect? Thanks again.

Installing speedtest-netperf (1.0.0-1) to root...

Collected errors:
 * check_conflicts_for: The following packages conflict with speedtest-netperf:
 * check_conflicts_for: 	speedtest * 
 * opkg_install_cmd: Cannot install package speedtest-netperf.

Update: I noticed that speedtest 0.9-7 was installed. Un-installing it and then installing speedtest-netperf works. Now running the following from SSH works as described in the OP:

speedtest-netperf.sh -H netperf-west.bufferbloat.net -p 1.1.1.1 --sequential

Feature request: I have sometimes wanted to measure the distribution of latency when the line is idle, without sending any data. Probably a -i option, like [ -s | -c | -i ]

That would display ordinary latency without any additional speedtest traffic. Obviously, all the latency measurements want to be formatted the same way, leaving off the speed measurements.

I don't know if it would be useful to include the CPU Load and Overhead numbers. Thanks!

2 Likes

surprise this day

opkg update
opkg install speedtest-netperf

that's all

after i can run :
speedtest-netperf.sh -H netperf-west.bufferbloat.net -p 1.1.1.1 --sequential

2 Likes

Great package, thank you!

Would be awesome to see a LuCI interface with this!

1 Like

The netperf-eu.bufferbloat.net server doesn't seem to work anymore... :frowning:

root@WRT:~# speedtest-netperf.sh -H netperf-eu.bufferbloat.net --sequential
2019-11-22 12:20:49 Starting speedtest for 60 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging gstatic.com.
Download and upload sessions are sequential, each with 5 simultaneous streams.
.
WARNING: netperf returned errors. Results may be inaccurate!

 Download:   0.00 Mbps
  Latency: [in msec, 0 pings, 0.00% packet loss]
 CPU Load: [in % busy (avg +/- std dev), 0 samples]
 Overhead: [in % used of total CPU available]
  netperf:   0.0
.
WARNING: netperf returned errors. Results may be inaccurate!

   Upload:   0.00 Mbps
  Latency: [in msec, 1 pings, 0.00% packet loss]
      Min:  28.342
    10pct:   0.000
   Median:   0.000
      Avg:  28.342
    90pct:   0.000
      Max:  28.342
 CPU Load: [in % busy (avg +/- std dev), 0 samples]
 Overhead: [in % used of total CPU available]
  netperf:   0.0

Do you guys know any other EU server for testing?

Hi @Kherby try netperf.richb-hanover.com

2 Likes

Thanks but this server doesn't seem to be EU based.

root@WRT:~# ping netperf.richb-hanover.com -c5
PING netperf.richb-hanover.com (23.226.232.80): 56 data bytes
64 bytes from 23.226.232.80: seq=0 ttl=50 time=116.458 ms
64 bytes from 23.226.232.80: seq=1 ttl=50 time=116.273 ms
64 bytes from 23.226.232.80: seq=2 ttl=50 time=116.690 ms
64 bytes from 23.226.232.80: seq=3 ttl=50 time=116.107 ms
64 bytes from 23.226.232.80: seq=4 ttl=50 time=116.540 ms

As far as I recall that server is on the east coast of the US, likely in Atlanta. But netperf-eu might be coming up again (if so probably after the weekend though).

why not try speed-test-netperf.sh -H netperf.bufferbloat.net --sequential
That is working for me without issues.