Upload 3x Faster than Download

Installing and Using OpenWrt
1

Hey, I'm looking for a bit of help with an issue I can't figure out. I'm new to OpenWRT, but have been eyeing it for a few years now. I'm looking to use it for whole network ad blocking and such; probably more use cases after I knock this one out.

My setup:
I have a fiber link with a separate ISP provided wireless AP. Using just their equipment I'm in the range of symmetrical 350 Mbps up/down.
I installed OpenWRT on a Raspberry Pi 3B+ and stuck it between the fiber "modem" (not sure the right term there) and the ISP provided wireless router.
I'm using the ISP wireless router for wireless network, and I attached a USB<->Ethernet adapter to the Pi (Realtek 8152 series).
In OpenWRT router settings I deleted the bridge, and setup one interface per network interface (WAN, LAN, WLAN). I setup the firewall rules to forward from LAN/WLAN to WAN, and all the DHCP/DNS things to make this connection work.
Looks something like this:
[Modem] <-> [Pi onboard ethernet] <-> [Pi usb/eth adapter] <-> [Wireless Router]

It does work, and the blocking is set in place; but I have an odd problem I can't figure out: I get about 3x the upload speed as I do my download speed (~ 50Mbps down, ~150 Mbps up).

After some fiddling I got the USB<->Ethernet adapter working faster. Can confirm via iperf3 speeds of about 250Mbps up and down to the pi itself. Using the speedtest CLI on the PI I can get speeds around 60 Mbps down 220 Mbps up (to the WAN side of things).

I've tried adjusting the software / hardware offloading, SQM settings, packet steering, and probably a few others I've forgotten. None of which seemed to have a noticeable effect. Does anyone have any ideas as to how to further troubleshoot this problem?

Including some important config files here:

root@OpenWrt:~# cat /etc/config/network 

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd76:2ceb:c81a::/48'

config interface 'wlan'
        option proto 'static'
        option device 'phy0-ap0'
        option ipaddr '192.168.11.1'
        option netmask '255.255.255.0'
        list dns '192.168.10.1'
        option delegate '0'

config interface 'wan'
        option proto 'dhcp'
        option device 'eth0'
        option peerdns '0'
        list dns '192.168.10.1'
        option delegate '0'

config interface 'lan'
        option proto 'static'
        option device 'eth1'
        option ipaddr '192.168.10.1'
        option netmask '255.255.255.0'
        list dns '192.168.10.1'
        option delegate '0'

config device
        option name 'eth0'
        option ipv6 '1'

config device
        option name 'eth1'
        option ipv6 '1'

config device
        option name 'phy0-ap0'
        option ipv6 '1'

root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option enabled '1'
        option interface 'eth0'
        option download '400000'
        option upload '400000'
        option qdisc 'cake'
        option script 'simplest_tbf.qos'
        option linklayer 'ethernet'
        option debug_logging '0'
        option verbosity '5'
        option overhead '44'

root@OpenWrt:~# cat /etc/config/firewall
                                               
config defaults                                                                                
        option input 'REJECT'                  
        option output 'ACCEPT'        
        option forward 'REJECT'    
        option synflood_protect '1'         
        option flow_offloading '1'
        option flow_offloading_hw '1'
                                               
config zone                   
        option name 'lan'                      
        option input 'ACCEPT'        
        option output 'ACCEPT' 
        option forward 'ACCEPT'
        list network 'lan' 
                                               
config zone                   
        option name 'wan'     
        option input 'REJECT'     
        option output 'ACCEPT'
        option forward 'REJECT'
        list network 'wan'    
        option masq '1'                        
                                               
config forwarding                       
        option src 'lan'
        option dest 'wan'  
                                               
config rule                        
        option name 'Allow-DHCP-Renew'                                                         
        option src 'wan'               
        option proto 'udp'            
        option dest_port '68'      
        option target 'ACCEPT'              
        option family 'ipv4'                
                                                                                               
config rule                                  
        option name 'Allow-Ping'                                                               
        option src 'wan'       
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule                   
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation' 
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config zone
        option name 'wlan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'wlan'

config forwarding
        option src 'wlan'
        option dest 'lan'

config forwarding
        option src 'lan'
        option dest 'wlan'
2

Not sure the pis before the 4B or the five are actually all that suited for your link rate, as they are limited to USB2 for ethernet, which is simply not fast enough for Gigabit ethernet...

3

Agreed, I should get a later model for even faster performance. Do you think this accounts for the disparity between the up and download speeds? That's the thing that really has me scratching my head here.

4

1 Gbps PHY connected to 480 Mbps USB 2.0, so it's 240 Mbps Ethernet

You are not alone, but it's definitely a Pi 3B+ issue:
https://forums.raspberrypi.com/viewtopic.php?t=208512

both ethernet are hooked into the USB 2.0 and it's been weird for years.

1 Like
5

Bah, good find. Hadn't seen that thread yet. I thought for sure I just had a setting wrong considering the decent upload speed... off to the Pi store... Thanks for your help!

6

Since you have set it up already, what happens if you configure sqm to shape each direction to 100Mbps? I wonder whether that will look balanced then?

7

Yeah, slightly different, I just used ethtool to do essentially that:

ethtool -s eth0 speed 100 duplex full

That actually evens it out: ~95Mbps up and down

2 Likes
8

Would 120 down and up be the theoretical limit?

9

Not sure, it is not that USB has no overhead itself, but #I was just curious whether things become more balanced once the USB is not saturated anymore...

10

You got me interested here as well. I set the interface back to 1Gbps full duplex, and used SQM to set limits at 120Mbps up and down, then turned off software and hardware flow offloading (for some reason this improved performance). My network is more busy now than during my original testing (wife is off work today), but that got me to ~85Mbps down and ~93Mbps up. Close enough to what I got setting the interface to 100Mbps

1 Like
11

Thank you for confirming.
In its day the 3B+ was good, my issue is these days a GL.inet Beryl AX completely wipes it out in speed, usability and cost.

1 Like
12

For anyone finding this in the future: I've upgraded to Raspberry Pi 4B using the same configuration as before. Additionally added wireguard (not OpenVPN) for a VPN tunnel and my speeds are now consistently 500Mbps up and down sometimes as high as 700Mbps. Note that this setup relies on the fact that I'm still using the fast, but locked down wifi router from ISP. Using the on-board wifi gets closer to 70Mbps.

13

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.