Ubiquiti EdgeRouter X, Loading OpenWrt and performance numbers

Hardware Questions and Recommendations
50

Yes, I believe it's mandatory in the DOCSIS 3.1 standard. On the other hand, the algorithm they use allows substantially more bufferbloat than does cake, on the third hand, it's still not a huge amount, 20ms or something like that is typical, enough that VOIP jitter buffers can usually handle it. Games might be a different story.

In any case, it seems like the two nice price points right now are espressobin at $50 + case + power supply for anything less than say 500Mbps, and x86 mini PC at around $200 for over 500 Mbps

For wifi consider an external access point, separate, replaceable, expandable, and you can situate it separately from the router to get better signal.

1 Like
51

@dlakelan ESPRESSObin is a temping alternative - going to consider it. How do you think it's going to hold up with long term reliability and support? I'd have to get a good heatsink for it

@slh While I understand what you're saying, how do you explain the people seeing these insane throughputs with SQM+SFE

Theory is nice, ain't it? However, I don't know of any proper deployment of any DOCSIS system that's not in a CableLabs' lab. I see about 250ms while loaded

I'm looking to go very deep down the rabbit hole regarding the manual fine-tuning of the latest and greatest QoS/SQM software

I need the first hop behind the DOCSIS 3.1 modem to be SQM protecting bandwidth sharing among my family

To bring this back on topic regarding ERX+SQM+SFE:

Is this Qualcomm Fast Path thing worth getting into with the ERX? If not, I cannot consider the ERX as a viable option. I read there's an issue with UDP offloading (is that issue with ingress or egress?)

From what I've read on the ubnt forums - the currently-used Cavium offloading is better.. but it can't help with SQM

52

In this case espressobin is the minimum and x86 if you can afford it. The hardware fastpath stuff simply isn't compatible with QoS at all

It's been running on my desk for several days continuously and isn't even very warm. I'm going to print a little case for it at my library mostly because I want to try their printer. Otherwise I'd probably throw it in a Ziploc tub and drill some holes in the case for airflow and shove it in my closet and forget it.

It's very low power at idle. Even running a 20Mbps 4k video stream is basically idle for this board.

53

As we run down the rabbit hole. I have an original APU board, the crappy one, with the Realtek nics, and only dual core. Looks like the Realtek nics, or something deep, is not taking advantage of the two cores. So a single core running cake maxes out around 300Mbps.

https://www.pcengines.ch/apu.htm

Stock APU board: http://www.dslreports.com/speedtest/44581168
APU board with Cake: http://www.dslreports.com/speedtest/44581272

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option interface 'eth1'
        option script 'simple.qos'
        option qdisc_advanced '0'
        option linklayer 'none'
        option enabled '1'
        option debug_logging '0'
        option verbosity '5'
        option qdisc 'cake'
        option download '300000'
        option upload '300000'
54

Thanks for the additional data point. I thought cake in general didn't take advantage of multiple cores?

1 Like
55

I think that's right at least for single instances. I'm not sure if separate instances might have separate threads?

1 Like
56

Keep in mind that a router doesn't only have to deal with cake, there's also the need to serve the incoming hardware IRQs, do the actual routing, NAT, wireless, etc. If you have a computationally heavy process pegging one core, it's good to have another one 'free' for the normal business (meaning at least two cores really make sense on a router, especially for VPN, cake or similar uses).

2 Likes
57

Thank you all for this thread. It made me go and buy one to replace my E4200v2 Linksys as my home router.

Here are some performance numbers regarding this router with OpenVPN.

Remote MacBook Pro (OpenVPN/iperf3 client) > (tcp over wan) via EdgeRouterX (OpenVPN server) > iMac at home (iperf3 server)

iperf3 -c 10.66.77.22
Connecting to host 10.66.77.22, port 5201
[  5] local 10.2.1.2 port 51385 connected to 10.66.77.22 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec  2.99 MBytes  25.0 Mbits/sec
[  5]   1.00-2.00   sec  3.07 MBytes  25.9 Mbits/sec
[  5]   2.00-3.00   sec  2.92 MBytes  24.5 Mbits/sec
[  5]   3.00-4.00   sec  3.11 MBytes  26.1 Mbits/sec
[  5]   4.00-5.00   sec  2.62 MBytes  21.9 Mbits/sec
[  5]   5.00-6.00   sec  2.79 MBytes  23.4 Mbits/sec
[  5]   6.00-7.00   sec  2.96 MBytes  24.8 Mbits/sec
[  5]   7.00-8.00   sec  2.81 MBytes  23.6 Mbits/sec
[  5]   8.00-9.00   sec  2.79 MBytes  23.4 Mbits/sec
[  5]   9.00-10.00  sec  2.96 MBytes  24.9 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  29.0 MBytes  24.3 Mbits/sec                  sender
[  5]   0.00-10.19  sec  28.4 MBytes  23.4 Mbits/sec                  receiver

iperf Done.

Note: I have FTTH connection at home.

For comparison, on the Linksys, I was getting slightly slower speeds.

[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.00  sec  25.9 MBytes  21.7 Mbits/sec                  sender
[  5]   0.00-10.14  sec  25.5 MBytes  21.1 Mbits/sec                  receiver

Next thing on my list, is looking into switching from OpenVPN to Wireguard for better performance.

58

The latest version of the Espressobin is available with an enclosure and a faster 1.2GHz processor from Amazon for $79.

59

One other person reported some instability on that v7 espressobin. I'd be interested to hear from anyone who tries it, in a separate thread.

60

I own an ERX and an Asus ac68u which has different architectures. The ac68u has a cpu which is similar to the one in espressobin hence I thought it might be relevant to bring previous posts I've made in another thread.

I wanted to compare sqm performance in these routers so I've run a couple of tests.

The asus ac68u has a dual core ARM which I've tried using frequencies between 400 mhz and 1400 mhz. Results are posted in this thread

61

This is a massive thread.. glad I found it..

Asking this here for it's relevance to this device.. When trying to make the serial connection..

https://www.db9-pinout.com/

I need pins 2, 3, and 5 to make the serial connection work right?

Not having success and wondering where I'm doing whatever wrong..

62

Connecting to a typical DB-9 will likely fry your router, as most computers and USB adapters with DB-9 connectors work either at TTL levels (5 V), or RS232 levels (nominally +/- 15 V). The latter will almost certainly fry your SoC's serial port at best, potentially your entire SoC. Even TTL levels can be deadly to SoCs.

Only use an adapter that uses 3.3 V logic levels. Many need to be configured with a jumper or solder bridge to set 3.3 V. Others are TTL- or RS232-only and are not appropriate.

Do not connect +3.3V, V+, Vcc, Vss, or the like between the router and the serial adapter; RX, TX, and ground are sufficient.

https://openwrt.org/toh/ubiquiti/ubiquiti_edgerouter_x_er-x_ka#serial

1 Like
63

Hi.

I have previously flashed an ERX using the process described on OP.
I just tried to flash another and now I can't connect to it in any way after reboot.

Here is the last logs:

ubnt@ubnt:/tmp$ add system image /tmp/erx-initramfs-factory.tar
Checking upgrade image...Done
Preparing to upgrade...Done
Clearing directory /var/cache/apt (332.0K)...Done
Copying upgrade image.../usr/bin/ubnt-upgrade: line 569: [: too many arguments
Done
Removing old image...Done
Checking upgrade image...Done
Copying config data...Done
Finishing upgrade...Failed to move new image

Any idea what can be done?

Nothing happens from eth1 to 4.
eth0 lights up if I insert cable, but I can't find which IP it is running on if there's any at all.

64

use tftpboot image using the serial console then write the sysupgrade again

65

Hi, I found this thread from the EdgeRouterX OpenWrt wiki page. A big thank you to dingbatca for making the openwrt-ramips-mt7621-ubnt-erx-initramfs-factory.tar bootstrap image available and the great instructions. The steps worked perfectly for me yesterday to set up an EdgeRouterX and then upgrade it to 18.06.2. This device is showing great performance using SQM on a 100/100 fibre internet connection. My previous device didn't have the CPU resources to shape traffic at this level.

I was wondering if the OpenWrt file kernel.bin file e.g. "openwrt-18.06.2-ramips-mt7621-ubnt-erx-initramfs-kernel.bin" can be used in the initial EdgeOS "add system image" step? It would be nice to use that file if it works for that purpose. I didn't want to possibly brick the device by just trying!

Thanks,
Tim

66

Another happy user here. Thanks @dingbatca for the great walkthrough!

On a gigabit cable connection (Comcast). With hardware offload, hitting in the mid 900's for download, right around 40 upload. Bufferbloat on downloads is staying under 20ms, and uploads ranges between 30 and 100.

With SQM enabled, hitting right at 200/40 but bufferbloat numbers aren't much better than the above (20-40 on average). So for now, staying with hardware offload.

67

For testing SQM you must set the speeds low enough that you have at least a little CPU idle time, try 180/35 and see what the delay under load looks like...

68

Try leaving the download shaper set at zero and keep using offloads for that, but try an upload only shaper; given your bufferbloat results this might both give better upload experience and still maintain high download bandwidth.

1 Like
69

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.