Thanks, that did it.
1 Like
Hi OpenWrt community,
first of all, thanks for this awesome project, it's incredible, I really love it.
I installed rpi-4-ext4-sysupgrade.img.gz from September 18, 2020 ( LuCI Master (git-20.262.48502-45c914a) / OpenWrt SNAPSHOT r14507-14247c82c0) and expanded the microSD using gparted.
My WAN port is on eth0 (Internal Pi LAN card), and the LAN port is on eth1 (External USB dongle). I use the Ugreen USB 3.0 to RJ45 Gigabit Ethernet Adapter dongle for that, requiring the kmod-usb-net-asix-ax88179 package to be installed.
I'm slightly disappointed (given an overall price of all components of about 100€) that apparently the Raspberry Pi 4 is currently not fast enough to handle a 1 Gigabit WAN connection (No-Pi: 1143 Megabits vs. With-Pi: 940 Megabits) CPU0 maxes out at 100%.
EDIT: I was wrong, more here
Raspberry Pi 4 B (4GB) = 58€
USB-C power supply = 5€
microSD 32GB = 6.5€
Armour case (fan-less Aluminium case) = 11€
UGREEN USB 3.0 to RJ45 Gigabit Adapter (ASIX887719) = 16€
------
Total: 96.5€
What I did:
- Overclocked the Pi
In the file /boot/config.txt
over_voltage=6
arm_freq=2000
-
Enabled Software flow offloading under Network -> Firewall
-
Enabled Receive Packet Steering (RPS) as suggested by dlakelan
In the file /etc/rc.local
echo 2 > /sys/class/net/eth1/queues/rx-0/rps_cpus
echo 1 > /sys/class/net/eth0/queues/rx-0/rps_cpus
My ISP gives me 1000 Megabit down, 50 Megabit up via Cable modem. When doing a speedtest with my laptop directly attached to my Cable modem I get about 1143/56. When using the Raspberry Pi 4 I "only" (sorry) get 940/54.
Is there anything I can do to fix that, or should I switch to x86 or what other OpenWrt-powered router can I buy to get the full speed? Thanks for a helpful reply.
1 Like
That's pretty much right at the top end of what you can get from a gigabit ethernet link. I'm not sure how you're getting 1143 on your laptop (unless the laptop and cable modem have got better than 1 gigabit network ports) but I would suspect it's not an accurate result. Rather than use a speedtest I would test using downloads, see if attaching the laptop directly still gives higher speeds.
Other than that, you can replace your current USB to ethernet adaptor to one with a Realtek chipset. From the testing done by dlakelan there's a significant drop in cpu usage when using an adapter using the RTL8153 chipset.
3 Likes
So the ASIX USB3 adapters are considered to be slower than the Realtek ones... @krazeh has the details 
As krazeh noticed, this is an ethernet limit. The gigabit ethernet interface in the RPi4B runs at 1000 Mbps gross rate which will allow the follwing measured goodput (assuming a cable/DOCSIS link and plain IPv4/TCP):
1000 * ((1500-20-20)/(1500+38)) = 949.28 Mbps...
your 940 is sufficiently close, indicating that this calculation is essentially showing the right result, even though your real link might carry slightly different overhead.
There is no way in hell (short of violating spec and compatibility by overclocking the ethernet clock) of pushing > 1000 goodput over a gigabit ethernet. So either laptop and modem use faster interfaces (like 2.5, 5, or even 10 Gbps Ethernet) or some sort of channel bonding (unlikely with a laptop) or via WiFI with sufficiently modern WiFI and essentially flawless radio transmission...
Get a router with faster ethernet interfaces, or, IMHO better ignore that additional Mbps your ISP provisions as the gimmick it is and just use the ~940 goodput, there will only be very few use cases where you actually would feel that difference...
4 Likes
Do you use SQM + cake? If so, what kind of bandwidth are you seeing?
Looks like gigabit fiber is coming to my area in a few months, and with a fairly large number of devices and users on the network I'm hoping I can enable it without too big of a bandwidth loss.
Hi,
thank you all for your helpful replies. You are correct. I was wrong, and blinded by fictive numbers my ISP (Vodafone) provides in its speedtests.
During my speedtests the maximum speed my xUbuntu network monitor indicator plugin shows is 985.41 Megabits. The Vodafone speedtest websites reports 1021 Megabits:
EDIT:
The following is outdated and wrong!
However, I also found a big issue with the onboard (aka internal) ethernet port on the Raspberry Pi 4. It's way slower (about 200 Megabits) than my external USB3 ethernet adapter.
My measurement setup using iperf3:
PC-A (iperf client) <- [external USB RJ45 adapter (LAN) - Pi 4 - internal ethernet port] (WAN)] <- PC-B (iperf server)
740-750 Megabits 
PC-A (iperf client) <- [internal ethernet port] (LAN) - Pi 4 - external USB RJ45 adapter (WAN) ] <- PC-B (iperf server)
930-940 Megabits 
P.S.: I also played around with running iperf server on the Pi4, but there was no noteworthy difference in speed.
I will keep the Pi as my main router for now. 
Not sure why you're getting a slower speed when using the onboard port as WAN, but it's not an issue with the port itself. If it was then the speed would be just as slow when you were using it as the LAN port.
interrupt stacking... ( agree, not the port itself )
True, but this is reproducible. For now I’m super happy and wasted only 1 day with it. 
EDIT: It's not!
In the future I might also play around with the recommended TP-Link UE300 USB 3 network adapter.
Sorry, I don’t use SQM and/or cake. I don’t even know what they are 
What's the CPU usage on the RPi4 like for each scenario?
1 Like
Hi,
it's embarrassing but my previous measurement results are not reproducible (so I edited my last post). Tonight I learned the difference between iperf3, the vodafone speedtest website, and a wget ISO download.
-
The internal/onboard (eth0) consumes less CPU0 (~15-30%) than my external UGREEN (ASIX887719) USB adapter (~100%) WITH iperf3. However, for ISO file downloads both are 100% which justifies my (need for) overclocking.
-
While the speedtest website is (sometimes) faster when eth1 (USB dongle) is the WAN interface, iperf3 is (always) faster on eth0 (internal/onboard ethernet) and ISO downloads are also (always) faster when eth0 is the WAN interface.
iperf3 (measuring for 60 seconds):
A) eth1 (USB) = WAN @ 1.5GHz = 924 Megabits - CPU0 100%
B) eth1 (USB) = WAN @ 2.0GHz = 965 Megabits - CPU0 100%
C) eth0 (Int.) = WAN @ 1.5GHz = 986 Megabits - CPU0 15-30%
D) eth0 (Int.) = WAN @ 2.0GHz = 983 Megabits - CPU0 15-30%
But be carefull, this is all iperf3 only, and the results suggest that there is no need for overclocking. However, when I download an ISO file, I do see a speed (879 Mbits @ 1.5 GHz, 965 Mbits @ 2.0GHz) and CPU (iperf3: 30%, ISO dl: 100%) difference!!!
As I'm only interested in fast downloads, and not what a speedtest reports, I switched my WAN interface to the internal (eth0) ethernet port of the Raspberry Pi 4 and overclocked it at 2.0GHz.
A) iperf3: eth1 (USB) = WAN @ 1.5GHz -> 924 Megabits
C) iperf3: eth0 (Int.) = WAN @ 1.5GHz -> 986 Megabits
B) iperf3: eth1 (USB) = WAN @ 2.0GHz -> 965 Megabits
D) iperf3: eth0 (Int.) = WAN @ 2.0GHz -> 983 Megabits
P.S.: I don't know whether reporting this makes any sense, but here you go:
cat /proc/interrupts (freshly rebooted before every test)
WAN=USB (eth1), after 1 minute iperf3
31: 60 889 0 0 0 GICv2 189 Level eth0
32: 13 524 0 0 0 GICv2 190 Level eth0
39: 1 505 759 0 0 0 BRCM STB PCIe MSI 524288 Edge xhci_hcd
WAN=Internal (eth0), after 1 minute iperf3
31: 228 942 0 0 0 GICv2 189 Level eth0
32: 366 016 0 0 0 GICv2 190 Level eth0
39: 839 902 0 0 0 BRCM STB PCIe MSI 524288 Edge xhci_hcd
Mmmh, you are measuring on different layers. Please only use measurements on one layer consistently, I recommend to use TCP/IP goodput, because that is easiest to repeatably measure, it is also the number that is returned from acceptable on-line speedtests like okkla's speedtest.net+ dslreports and breitbandmessung.de.. Iperf also returns that value and it is easy to calculate for any data transfer by hand simply by dividing the volume of bytes by the duration of the transfer.
1 Like
also...
opkg install sysstat
mpstat -A 2
will shed the best light at a system resource allocation level... ( i'd actually really like to see the difference )
I wanted to see what I could do with the Raspberry, so I stumbled upon OpenWRT. Currently I’m using pfSense running on a Fitlet mini PC, and that firewall solution has been rock solid for over 4 years now. My uptime is normally over a year. I’m a Cisco network engineer, and I’ve run Cisco routers or firewalls for my Internet access for decades, so when I switched to pfSense, I was very pleasantly surprised with its features and stability. I run the update once a year on pfSense, and the update process takes just a few minutes. I don’t have to do anything special, as all the settings survive.
So, running OpenWRT on the Raspberry Pi is an interesting project for me. I may even replace pfSense with it, but I have a few questions and concerns after I’ve read this entire thread.
-
What’s the recommended USB3-to-GigabitEthernet adapter? I hear that the one based on the Realtek 8152 chipset is recommended. Can anyone recommend the actual adapter?
-
I have a 2.5 Gbps USB3 adapter based on the Realtek 8156 chipset, and I couldn’t get it to be installed as an interface under OpenWRT even after I installed the Realtek 8152 driver. Does anyone know if there’s a driver for the 8156 chipset to be used as a second Ethernet interface in OpenWRT?
If it’s possible to use 2.5 Gbps USB interfaces in OpenWRT, can the Raspberry Pi 4b route at speeds higher than 1 Gbps? Has anyone tried this with two USB3 2.5 Gbps adapters, since some newer cable modems now come with 2.5 Gbps Ethernet interfaces?
-
I don’t quite understand the difference between a snapshot image and a mainstream image. Can someone explain?
-
What’s the difference between the squashfs and ext4 images?
-
I’ve read in this thread that the upgrade to another snapshot image is complicated and the settings do not survive. How will the upgrade procedure be different when when the Raspberry Pi 4b becomes part of the mainstream image?
-
What’s the recommended size of the microSD card in order to run additional packages / services inder OpenWRT? I’m
interested in wireguard, pihole, squid, etc. -
Is there a USB3 based Wi-Fi dongle that can be used with OpenWRT running on the Raspberry Pi 4b to turn it into a decent wireless AC router with about 500-600
Mbps throughput on the 5Ghz radio? -
Is it possible to have a console port access with the OpenWRT running on the Raspberry Pi 4b? If so, how can this be done?
Thank you very much!
here are my answers to few of your questions:
#3 - snapshot is daily update, while stable gets update less frequently
I have been using snapshots w my rpi4 and follow the daily updates for the none core pkgs, and do full image update once a while (bi-monthly)
#4 - Ext4 vs SquashFS
#5 - the way for upgrading is the same regardless of image, create backup, save pkg list and anything extra you did, upgrade and restore those extra pkgs, configs, users etc etc, (I scripted it)
#8 never required a serial console, think about it, you have ssh, if that fails just get the SD to an other system and correct issues
1 Like
#8 normally documented UART pins work out of the box for console access. They are NOT 5v tolerant... If you instantiate other/additional UART pins via devicetree-overlay, alterations need to be made to cmdline.txt and inittab also.
So, I've compiled my first 64-bit image and even got the Plugable 2.5 Gbps rtl8156 based chipset USB to Ethernet adapter to load a driver. So, I now have two Ethernet interfaces on the Raspberry Pi 4B: on-board 1 Gbps and a USB3-based 2.5 Gbps.
I've been reading about OpenWRT for a few days now. There's a lot of information to process.
-
Can someone explain to me how I would be updating the image that I compiled the other day? I'm going to replicate my pfSense configuration in OpenWRT and try it as my router/firewall. However, since the 64-bit image is not yet included in a stable release, I need to understand how to update it later on. I can compile my own sysupdate image, but how do I install it and what happens with my settings after I install it? Will my settings survive? Will my packages get upgraded and their settings survive?
-
Once Raspberry Pi 4B is included in a stable release, how would software updates happen then? Is there a way to update without blowing away the settings and packages? I'm hoping that the update mechanism in OpenWRT works kind of like this is done in pfSense, where you simply choose the update image from the GUI menu, click OK, and everything gets automatigically updated or you could run a software update from a contextual text menu via SSH or serial console in pfSense.
Now that I've read up on the OpenWRT upgrade procedure, I'm a little concerned. This is a pretty involved process, especially if one installs additional packages. I guess I can keep building my own image with the packages pre-built into the system image, but it takes some time.
So, provided this process is not as simple as the pfSense upgrade process, which takes less than 5 minutes of downtime, how often do you guys upgrade your OpenWRT systems? Do you upgrade with every stable release? How often are stable releases published? How are newly discovered vulnerabilities addressed? If there's a vulnerability patch published, is it a new stable firmware that needs to be updated or are there patches issued that can be installed without blowing away the entire system?
Thank you.
This is how I do it on all my OpenWRT devices:
When there's a new official service release I make a backup, just in case. Then I do the upgrade while keeping the current settings, then I reinstall all the packages that don't come with the official image. The config files will be there for the packages to work just fine.It can even be scripted if you'd like.
For new major versions I just redo all my settings. Less headaches and possible problems.
Of course the Pi4 won't have an official image until 20.x releases, but this method works with your custom made images.
1 Like