Recommended router for asymmetric gigabit speed

No, I mean raspberry pi 4

For information on Raspberry Pi 4 USB dongle selection and SQM performance, see https://forum.openwrt.org/t/rpi4-routing-performance-numbers/53996

I would still recommend the 4GB NanoPi R4S to the OP over a RPI4, because the NanoPi R4S already comes with a second Gig Ethernet port and a really nice metal case/heat sink. No USB dongle required. Easy. No issues. Done.

That's not to say the RPI4 cannot be made into a workable Gig SQM solution with the right USB dongle. I'm just trying to keep it simple.

6 Likes

only saw mr van tech's video yesterday... was really impressed with the layout/form factor and openvpn performance of the r4s

it's definitely a better board from a purely hardware perspective

but... if you dont need openvpn or really neat small form factor then they come out about equal imho... if not the rpi4 slightly on top due to being available(yes... yes... shortage etc.) everywhere, lower thermals? and better code support in general

i'll be keeping a close eye rk boards... specifically thermals and general support status

here is one up and comer in the works, getting extremely close to competing with larger solutions (i.e. mochabin , turris2022 etc.)

raxda rock3b link

2 Likes

If you are running OpenWrt on bare metal the better code support on RasPi is basically gone, and 4 is notoriously known for running hot and need cooling solution to not throttle.

Radxa is also working on E23 and E25, which are dual NIC board with single/dual 2.5GbE.

1 Like

Indeed, I doubt very much CS:GO is the most popular esport in the world while having netcode that can't fairly adjudicate delays that are 4% of an eyeblink

maybe if you're watching video on it but 100% A non issue in routing for current board revisions.

Well yeah if you only use it for routing. But that's a waste of hardware IMO since you can run Proxmox or docker on it.

Which in general is a tempting/attractive proposition, given that routers are on 24/7. But you are putting all of your eggs in one basket, security wise. What will you do if you long running (and npt properly snapshotted) container needs another day of run time, but the router part wants a fix for a zero-day exploit that was just revealed immediately? Yes, this is a bit contrived, but it shows the opposing forces at work IMHO.

Problem with running OpenWRT containerized is you lose flow offloading at this time -- which may not be an issue when using SQM. But also, I find in practice that while my router has plenty of headroom to run other services besides routing, I prefer not to do it as it introduces runtime risks and maintenance issues by linking concerns that ought to be unrelated, e.g. it can create logistical problems because of gratuitous dependencies of unrelated services on the same hardware. (Edit: basically more or less what @moeller0 said.)

I do however run LXC containers in OpenWRT (rootfs's under LVM2 storage), for pure network services that are already completely dependent on the router being functional (and therefore won't be gratuitously affected by router maintenance), such as AdGuard Home and some MQTT related services, which I don't want to get wiped and have to restore when I update OpenWRT.

1 Like

+1 RT3200 - I have three. In another OpenWrt thread it was concluded it can handle 1Gbit SQM.

3 Likes

I had exactly this conundrum a few months back when upgrading from VDSL to FTTP with 900/110 down/up speeds. I read a lot of threads in this forum which helped and confused in almost equal measure.

I've always been a fan of Linksys WRT series routers but I decided to go another, recycling route by buying a refurbished Dell OptiPlex 3020 SFF with an i5 and a couple of HP NC110T Gigabit Ethernet cards - I'm not a fan of Realtek cards having had problems in the past and these HP cards are Intel based and designed for servers so are pretty much bulletproof. Total cost was £100/$120.

The i5 is overkill so you could get an i3 for a small saving. I've never seen the load go above 3% even with 2 heavy gamers in the house. I use a mesh system for wifi to the lack of this isn't a problem, but you could replace one of the HPs ethernets with a wifi card and use the Realtek if you needed to,

OpenWrt runs like a dream, and I'll never need to buy another router :smile:

1 Like

Here are solutions similar to the one you recommend (refurbished PC) and cost about the same:

You just have to buy any of these genuine and used NICs from eBay:

Intel NIC Genuine vs Fake:

Another +1 for the RPi 4 here. Put it in a nice passive case (Argon Neo, Flirc, any of the aluminum "armor cases" to name just a few) and you'll never have an issue with thermals in any routing application I'm aware of. I have the onboard gigabit port connected to my LAN and am using a TP-Link UE300 for WAN with absolutely rock-solid stability and full gigabit throughput. Wireless connectivity is provided via a TP-Link RRE603X which I purchased for $60 and have located centrally in the home with great coverage.

With that said, I do have a NanoPi R4S on its way though with the state of the world it might be another month or two before it reaches my doorstep. If they only had better US-based distribution around that same $80-$90 price point I think that would instantly become the no-brainer choice for a lot of setups.

@Sentenzo is right, maybe his delivery in truth ruffles some fans of the pi4. PI and A72 derivatives are just not cost efficient. For me, I ripped out some PI4 because they are simply too slow and replace them with x86. @anon50098793 already documented the limitations of the core/ethernet/networking bindings of the PI4 that increase latency when bandwidth increases past 600mb. Translated into real world, @Sentenzo documented the 12ms --> 19 ms (almost 100% increase in latency) jump in his competitive games. STEAM/valve which owns CS GO is not some small company with inexperienced network engineers.

Raleigh, NC is already active with 2.5 gigabit and 5 gigabit options on ATT Fiber as of November 2021. Pretty soon you'll see it rollout nationwide in 2022. To put it in context for worldwide people, ATT Fiber has a pretty massive footprint in USA -- either #1 or #2 ONT fiber ISP.

TL:DR: New information from real world usage = people should recognize the very real tradeoffs with PI4 and derivatives.

1 Like

Let's break this down:
100*19/12 = 158.33
That is even less than a 66%-age point increase... and this takes his root cause analysis at face value.

Yes, but that will not retroactively fix a netcode design intolerant to unequal static latencies and more importantly unequal jitter... I do not want to diss CS:GO here, but it clearly is a game that often is cited when network issues are brought up by gamers (could well be, that it is just played by more players than other FPS games, which would explain the apparent incidence rate).

Sure, but it still is one of the cheapest way to get a router that (once competently configured and not overburdened with additional chores) will allow you relative low latency under load routing on a budget. Sure there are other options available, all also in the ARM camp though.

If we are in the "bragging rights department" ISPs in Switzerland and France deploy ~10Gbps XGS-PON, while Init7 a small swiss ISP even offers 25Gbps for consumers. But that is pretty much irrlevant for the billions of users still on <= 1 Gbps links, no?

2 Likes

I was curious so I went ahead and did an Ookla speedtest and Waveform bufferbloat tests both with SQM off and with SQM configured. Router is a 2GB Pi 4 with TP-Link UE300 USB adapter connected to WAN. WAN connection is CenturyLink fiber, advertised as symmetric gigabit service.

No SQM

SQM enabled (piece_of_cake, 880000 kbit/s down, 650000 kbit/s up)

Those seem like respectable results to me? What am I missing in this conversation?

2 Likes

There you go, bringing facts about the real world into the conversation, Heretic!

4 Likes

I made a few additional tweaks to my configuration this morning and got even better results. I'm really impressed!

  • CPU overclocked to 2GHz
  • Software offloading disabled
  • Packet steering enabled
  • irqbalance installed and enabled (I feel like there may still be some room for improvement here; cores 2 and 3 are still lightly loaded during Waveform tests though this may be by design/due to hardware limitations)

SQM off

Ookla speedtest: https://www.speedtest.net/result/12507930601

  • 2ms ping
  • 927.14 Mbps down
  • 928.83 Mbps up

SQM on (piece_of_cake @ 850 Mbps down, 850 Mbps up)

Ookla speedtest: https://www.speedtest.net/result/12507942606

  • 2ms ping
  • 794.79 Mbps down
  • 800.76 Mbps up

Waveform: https://www.waveform.com/tools/bufferbloat?test-id=e4d378fc-2024-46a9-adac-bdfb1c3c71f0

  • 14 ms unloaded latency
  • +0 ms download active
  • +0 ms upload active
  • 808.3 Mbps down
  • 799.1 Mbps up
2 Likes

I am very frustrated with tests that don't test up and down simultaneously.

I am with you on that. So I hacked @richb-hanover-priv's betterspeedtest to use OOkla's speedtest.nat as load generator:

#!/bin/sh

# use Ookla's speedtest.net infrastructure to generate loads and ICMP echos ("pings")
# to concurrently assess the latency under load behaviour of a link.
# This is a crude hack around the issue that netperf/iperf[|2|3] reflectors are
# really rare on the internet, while speedtest.net nodes are rather ubiquitious...
# to implement high resolution pings this should be called under "sudo"


# betterspeedtest.sh - Script to simulate http://speedtest.net
# Start pinging, then initiate a download, let it finish, then start an upload
# Output the measured transfer rates and the resulting ping latency
# It's better than 'speedtest.net' because it measures latency *while* measuring the speed.

# Usage: sh betterspeedtest.sh [-4 -6] [ -H netperf-server ] [ -t duration ] [ -p host-to-ping ] [ -n simultaneous-streams ]

# Options: If options are present:
#
# -H | --host:   DNS or Address of a netperf server (default - netperf.bufferbloat.net)
#                Alternate servers are netperf-east (east coast US), netperf-west (California), 
#                and netperf-eu (Denmark)
# -4 | -6:       enable ipv4 or ipv6 testing (ipv4 is the default)
# -t | --time:   Duration for how long each direction's test should run - (default - 60 seconds)
# -p | --ping:   Host to ping to measure latency (default - gstatic.com)
# -n | --number: Number of simultaneous sessions ()

# Copyright (c) 2014 - Rich Brown rich.brown@blueberryhillsoftware.com
# GPLv2

# Summarize the contents of the ping's output file to show min, avg, median, max, etc.
#   input parameter ($1) file contains the output of the ping command

summarize_pings() {     
  
  # Process the ping times, and summarize the results
  # grep to keep lines that have "time=", then sed to isolate the time stamps, and sort them
  # awk builds an array of those values, and prints first & last (which are min, max) 
  # and computes average.
  # If the number of samples is >= 10, also computes median, and 10th and 90th percentile readings
  sed 's/^.*time=\([^ ]*\) ms/\1/' < $1 | grep -v "PING" | sort -n | \
  awk 'BEGIN {numdrops=0; numrows=0;} \
    { \
      if ( $0 ~ /timeout/ ) { \
          numdrops += 1; \
      } else { \
        numrows += 1; \
        arr[numrows]=$1; sum+=$1; \
      } \
    } \
    END { \
      pc10="-"; pc90="-"; med="-"; \
      if (numrows == 0) {numrows=1} \
      if (numrows>=10) \
      {   ix=int(numrows/10); pc10=arr[ix]; ix=int(numrows*9/10);pc90=arr[ix]; \
        if (numrows%2==1) med=arr[(numrows+1)/2]; else med=(arr[numrows/2]); \
      }; \
      pktloss = numdrops/(numdrops+numrows) * 100; \
      printf("  Latency: (in msec, %d pings, %4.2f%% packet loss)\n      Min: %4.3f \n    10pct: %4.3f \n   Median: %4.3f \n      Avg: %4.3f \n    90pct: %4.3f \n      Max: %4.3f\n", numrows, pktloss, arr[1], pc10, med, sum/numrows, pc90, arr[numrows] )\
     }'
}

# Print a line of dots as a progress indicator.

print_dots() {
  while : ; do
    printf "."
    sleep 1s
  done
}

# Stop the current print_dots() process

kill_dots() {
  # echo "Pings: $ping_pid Dots: $dots_pid"
  kill -9 $dots_pid
  wait $dots_pid 2>/dev/null
  dots_pid=0
}

# Stop the current ping process

kill_pings() {
  # echo "Pings: $ping_pid Dots: $dots_pid"
  kill -9 $ping_pid 
  wait $ping_pid 2>/dev/null
  ping_pid=0
}

# Stop the current pings and dots, and exit
# ping command catches (and handles) first Ctrl-C, so you have to hit it again...
kill_pings_and_dots_and_exit() {
  kill_dots
  echo "\nStopped"
  exit 1
}

# ------------ Measure speed and ping latency for one direction ----------------
#
# Call measure_direction() with single parameter - "Download" or "  Upload"
#   The function gets other info from globals determined from command-line arguments

measure_direction() {

  # Create temp files
  PINGFILE=$( mktemp /tmp/ping_output.XXXXXX ) || exit 1
  SPEEDFILE=$( mktemp /tmp/ooklaspeedtest_output.XXXXXX ) || exit 1
  DIRECTION=$1

  # Start dots
  print_dots &
  dots_pid=$!
  # echo "Dots PID: $dots_pid"

  # Start Ping
  if [ $TESTPROTO -eq "-4" ]
  then
    ping  -i 0.2 $PINGHOST > $PINGFILE &
  else
    ping6 -i 0.2 $PINGHOST > $PINGFILE &
  fi
  ping_pid=$!
  # echo "Ping PID: $ping_pid"
  
  OOKLA_DIR_ARG=""
  
  # Start netperf with the proper direction
  if [ "$DIRECTION" = "Download" ]; then
    OOKLA_DIR_ARG="--no-upload"
  fi

  if [ "$DIRECTION" = "Upload" ]; then
    OOKLA_DIR_ARG="--no-download"
  fi

  if [ "$DIRECTION" = "Both" ]; then
    OOKLA_DIR_ARG=""
  fi

  
  if [ "$MAXSESSIONS" -eq "1" ]; then
    OOKLA_SESS_ARG="--single"
  else
  # otherwise default to whatever ookla uses, the cli app has no control for this
    OOKLA_SESS_ARG=""
  fi
  
  
  
#  # Start $MAXSESSIONS datastreams between netperf client and the netperf server
#  # netperf writes the sole output value (in Mbps) to stdout when completed
#  for i in $( seq $MAXSESSIONS )
#  do
#    netperf $TESTPROTO -H $TESTHOST -t $dir -l $TESTDUR -v 0 -P 0 >> $SPEEDFILE &
#    # echo "Starting PID $! params: $TESTPROTO -H $TESTHOST -t $dir -l $TESTDUR -v 0 -P 0 >> $SPEEDFILE"
#  done
  
  # Wait until each of the background netperf processes completes 
  # echo "Process is $$"
  # echo `pgrep -P $$ netperf `



  if [ $DIRECTION = "Both" ]; then
    speedtest --no-upload  ${OOKLA_SESS_ARG} >> ${SPEEDFILE} & OOKLA_DOWN_APP_PID=$!
    #echo "Started PID ${OOKLA_DOWN_APP_PID} params: $TESTPROTO -H $TESTHOST -t $dir -l $TESTDUR -v 0 -P 0 >> $SPEEDFILE"
    echo "Started PID ${OOKLA_DOWN_APP_PID} command: speedtest-cli --no-upload  ${OOKLA_SESS_ARG} >> ${SPEEDFILE}"

    speedtest --no-download  ${OOKLA_SESS_ARG} >> ${SPEEDFILE} & OOKLA_UP_APP_PID=$!
    #echo "Started PID ${OOKLA_UP_APP_PID} params: $TESTPROTO -H $TESTHOST -t $dir -l $TESTDUR -v 0 -P 0 >> $SPEEDFILE"
    echo "Started PID ${OOKLA_UP_APP_PID} command: speedtest-cli --no-download  ${OOKLA_SESS_ARG} >> ${SPEEDFILE}"
    wait ${OOKLA_UP_APP_PID}
    wait ${OOKLA_DOWN_APP_PID}
  else
    speedtest ${OOKLA_DIR_ARG} ${OOKLA_SESS_ARG} >> ${SPEEDFILE} & OOKLA_APP_PID=$!
    #echo "Started PID ${OOKLA_APP_PID} params: $TESTPROTO -H $TESTHOST -t $dir -l $TESTDUR -v 0 -P 0 >> $SPEEDFILE"
    echo "Started PID ${OOKLA_APP_PID} command: speedtest-cli ${OOKLA_DIR_ARG} ${OOKLA_SESS_ARG} >> ${SPEEDFILE}"

    # this will all run inside a Python instance which will be hard to catch so rather collect the PID directly
    wait ${OOKLA_APP_PID}
  fi


  if [ $DIRECTION = "Both" ]; then
    echo ""
    echo $( grep -e Download $SPEEDFILE )
    echo $( grep -e Upload $SPEEDFILE )

  else
    # Print TCP Download speed
    echo ""
    #echo " $1: " $( awk '{s+=$1} END {print s}' $SPEEDFILE ) Mbps
    #echo " $1: " $( grep -e $1 $SPEEDFILE ) Mbps
    echo $( grep -e $1 $SPEEDFILE )
  fi


  # When netperf completes, stop the dots and the pings
  kill_pings
  kill_dots

  # Summarize the ping data
  summarize_pings $PINGFILE

#cat $PINGFILE
  rm $PINGFILE
  rm $SPEEDFILE
}

# ------- Start of the main routine --------

# Usage: sh betterspeedtest.sh [ -4 -6 ] [ -H netperf-server ] [ -t duration ] [ -p host-to-ping ] [ -n simultaneous-sessions ]

# “H” and “host” DNS or IP address of the netperf server host (default: netperf.bufferbloat.net)
# “t” and “time” Time to run the test in each direction (default: 60 seconds)
# “p” and “ping” Host to ping for latency measurements (default: gstatic.com)
# "n" and "number" Number of simultaneous upload or download sessions (default: 5 sessions;
#       5 sessions chosen empirically because total didn't increase much after that number)

# set an initial values for defaults
TESTHOST="netperf.bufferbloat.net"
TESTDUR="60"
PINGHOST="gstatic.com"
MAXSESSIONS="5"
TESTPROTO="-4"

# read the options

# extract options and their arguments into variables.
while [ $# -gt 0 ] 
do
    case "$1" in
      -4|-6) TESTPROTO=$1 ; shift 1 ;;
        -H|--host)
            case "$2" in
                "") echo "Missing hostname" ; exit 1 ;;
                *) TESTHOST=$2 ; shift 2 ;;
            esac ;;
        -t|--time) 
          case "$2" in
            "") echo "Missing duration" ; exit 1 ;;
                *) TESTDUR=$2 ; shift 2 ;;
            esac ;;
        -p|--ping)
            case "$2" in
                "") echo "Missing ping host" ; exit 1 ;;
                *) PINGHOST=$2 ; shift 2 ;;
            esac ;;
        -n|--number)
          case "$2" in
            "") echo "Missing number of simultaneous sessions" ; exit 1 ;;
            *) MAXSESSIONS=$2 ; shift 2 ;;
          esac ;;
        --) shift ; break ;;
        *) echo "Usage: sh betterspeedtest.sh [-4 -6] [ -H netperf-server ] [ -t duration ] [ -p host-to-ping ] [ -n simultaneous-sessions ]" ; exit 1 ;;
    esac
done

# Start the main test

if [ $TESTPROTO -eq "-4" ]
then
  PROTO="ipv4"
else
  PROTO="ipv6"
fi
DATE=$( date "+%Y-%m-%d %H:%M:%S" )
echo "$DATE Testing against $TESTHOST ($PROTO) with $MAXSESSIONS simultaneous sessions while pinging $PINGHOST ($TESTDUR seconds in each direction)"

# Catch a Ctl-C and stop the pinging and the print_dots
trap kill_pings_and_dots_and_exit HUP INT TERM

measure_direction "Download" 
measure_direction "Upload" 
measure_direction "Both" 

exit 0

which requires a specific python speedtest-cli variant (will not work with the official Ookla CLI as far as I can tell since that does not allow to selectively select the direction to test).

But from long time exposure to RRUL, I actually think that there is another test worth doing, saturate one direction and then "pump" the other (something like 100% load, 50% load 0% load, 100%, 50%, 0%, ... for a few seconds each, then repeat with the constant and pumped load direction reversed) that would also test the transients of load shifting (which might be more realistic that continously loading both directions of a link)