Netgear R7800 exploration (IPQ8065, QCA9984)

For Developers
1271

I did a little experiment today, where I set the affinity for eth0, eth1 and wlan0 (5Ghz) to 1 and the affinity for wlan1 to 2. I'm using the init script that runs irqbalance first, and it puts various other things across the cores, but I've seen a marked improvement in wan to lan throughput and also in 5Ghz wifi throughput. Before I was getting around 180-220 Mbps upstairs on 5Ghz whereas now I'm getting a consistent 320Mbps.

YMMV of course, but my suspicion that having wlan0 and eth0 on different cores would lead to a bottleneck seem to be borne out.

1272

I've noticed that switch LAN ports are numbered backwards in LuCI, i.e. numbering is not corresponding to the actual physical port labels.
Here is a patch for R7800, not sure if other Netgear routers are affected as well, perhaps somebody can test.

diff --git a/target/linux/ipq806x/base-files/etc/board.d/02_network b/target/linux/ipq806x/base-files/etc/board.d/02_network
index f5daf62653..7ea1110da0 100755
--- a/target/linux/ipq806x/base-files/etc/board.d/02_network
+++ b/target/linux/ipq806x/base-files/etc/board.d/02_network
@@ -16,12 +16,15 @@ compex,wpq864 |\
 netgear,d7800 |\
 netgear,r7500 |\
 netgear,r7500v2 |\
-netgear,r7800 |\
 qcom,ipq8064-ap148 |\
 tplink,vr2600v)
        ucidef_add_switch "switch0" \
                "1:lan" "2:lan" "3:lan" "4:lan" "6@eth1" "5:wan" "0@eth0"
        ;;
+netgear,r7800)
+       ucidef_add_switch "switch0" \
+               "1:lan:4" "2:lan:3" "3:lan:2" "4:lan:1" "6@eth1" "5:wan" "0@eth0"
+       ;;
 linksys,ea8500)
        hw_mac_addr=$(mtd_get_mac_ascii devinfo hw_mac_addr)
        ucidef_add_switch "switch0" \
1273

Hello,
I'm new to the forum but a long time openwrt/LEDE/openwrt user.

Currently I'm on a Netgear Nighhawk r7800 with 18.06.0-rc1. I don't know what's going on but I'm unhappy with performance. I see lots of 100% cpu spikes. I see max 800mbs of lan-lan from router to wired linux client using iperf. I see max of 300mbs lan-wan speed using multitude of different internet speed targets and curls to fast download targets. I've tested parallel d/ls and see the same speed cap. I'm on Ting Communications 1 gig-e fiber. I've tried tons of buffer tunes with little to adverse affect.

My setup is simple with a few wired and wireless clients (all 5ghz). I'm not concerned with wifi speed or bufferbloat as the great majority of my activity is single thread downloads.

Does anyone have any insight or idea?

Thanks!

1274

Of course my router wants to make a fool of me :wink:

# iperf -c 192.168.0.10 -t 10
------------------------------------------------------------
Client connecting to 192.168.0.10, TCP port 5001
TCP window size: 43.8 KByte (default)
------------------------------------------------------------
[  3] local 192.168.0.1 port 46364 connected with 192.168.0.10 port 5001
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  1.10 GBytes   946 Mbits/sec
1275

I finally found a public iperf site ... my stats looks descent

# iperf -c iperf.volia.net -r -P 4
------------------------------------------------------------
Server listening on TCP port 5001
TCP window size:  512 KByte (default)
------------------------------------------------------------
------------------------------------------------------------
Client connecting to iperf.volia.net, TCP port 5001
TCP window size:  512 KByte (default)
------------------------------------------------------------
[  8] local 64.98.123.130 port 57340 connected with 82.144.193.18 port 5001
[  5] local 64.98.123.130 port 57336 connected with 82.144.193.18 port 5001
[  6] local 64.98.123.130 port 57338 connected with 82.144.193.18 port 5001
[  7] local 64.98.123.130 port 57342 connected with 82.144.193.18 port 5001
[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-10.0 sec   176 MBytes   148 Mbits/sec
[  7]  0.0-10.1 sec   178 MBytes   148 Mbits/sec
[  8]  0.0-10.1 sec   174 MBytes   145 Mbits/sec
[  5]  0.0-10.1 sec   181 MBytes   150 Mbits/sec
[SUM]  0.0-10.1 sec   709 MBytes   591 Mbits/sec

I'm curious as to why txqueulen isn't available in OpenWrt

1276

Wait for the next RC as that might have Kernel 4.14.51, 4.14.50 has some issues with single stream performance lan to wan.

Also, make sure flow offload is enabled, I can push roughly 800Mbps lan to wan, any more will need hardware accel.

1 Like
1277

Both master and 18.06 have been updated to kernel 4.14.52 today.

2 Likes
1278

Nat hw accel is not equal to flow offload??

1279

Best to ask on the R7800 exploration or the flow offload threads, this thread is JUST for this particualr build.

1280

You mixed the threads.
This thread is for generic R7800 questions... (my build thread is separate)

HW acceleration in separate add-on for some chips (not ipq806x), the flow offloading has also the software part that is supported by kernel and works with R7800.

1281

Using the @hnyman master-r7360 build scripts I modified the flash memory partitions and built a new firmware. Created a 7MB read/write "jffs" partition to replace the read-only "reserve" one.

root@R7800RT1:~# cat /etc/banner|grep Open
OpenWrt SNAPSHOT, r7374-afac2a2

root@R7800RT1:~# uname -mrvos
Linux 4.14.52 #0 SMP Tue Jul 3 05:24:45 2018 armv7l GNU/Linux

root@R7800RT1:~# dmesg|grep '\-0x000'
[ 1.062422] 0x000000000000-0x000000c80000 : "qcadata"
[ 1.089929] 0x000000c80000-0x000001180000 : "APPSBL"
[ 1.099241] 0x000001180000-0x000001200000 : "APPSBLENV"
[ 1.100870] 0x000001200000-0x000001340000 : "art"
[ 1.106208] 0x000001340000-0x000001480000 : "artbak"
[ 1.110986] 0x000001480000-0x000001880000 : "kernel"
[ 1.120740] 0x000001880000-0x000007900000 : "ubi"
[ 1.286099] 0x000007900000-0x000008000000 : "jffs" <-- Replaces the old "reserve"

root@R7800RT1:~# cat /proc/mtd

dev:    size   erasesize  name
mtd0: 00c80000 00020000 "qcadata"
mtd1: 00500000 00020000 "APPSBL"
mtd2: 00080000 00020000 "APPSBLENV"
mtd3: 00140000 00020000 "art"
mtd4: 00140000 00020000 "artbak"
mtd5: 00400000 00020000 "kernel"
mtd6: 06080000 00020000 "ubi"
mtd7: 00700000 00020000 "jffs"

update opkg and prepare to install a package:

root@R7800RT1:~# opkg update
Downloading http://downloads.openwrt.org/snapshots/targets/ipq806x/generic/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_core
Downloading http://downloads.openwrt.org/snapshots/targets/ipq806x/generic/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/base/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_base
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/base/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/luci/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_luci
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/luci/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/packages/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_packages
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/packages/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/routing/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_routing
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/routing/Packages.sig
Signature check passed.
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/telephony/Packages.gz
Updated list of available packages in /var/opkg-lists/openwrt_telephony
Downloading http://downloads.openwrt.org/snapshots/packages/arm_cortex-a15_neon-vfpv4/telephony/Packages.sig
Signature check passed.

Install the nand-utils package:

root@R7800RT1:~# opkg install nand-utils
Installing nand-utils (2.0.2-1) to root...
Downloading http://downloads.openwrt.org/snapshots/targets/ipq806x/generic/packages/nand-utils_2.0.2-1_arm_cortex-a15_neon-vfpv4.ipk
Configuring nand-utils.

Perform an erase and a jffs2 format of the mtd7 partition:

root@R7800RT1:~# flash_erase -j /dev/mtd7 0 0
Erasing 128 Kibyte @ 0 -- 0 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 20000 -- 1 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 40000 -- 3 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 60000 -- 5 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 80000 -- 7 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ a0000 -- 8 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ c0000 -- 10 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ e0000 -- 12 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 100000 -- 14 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 120000 -- 16 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 140000 -- 17 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 160000 -- 19 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 180000 -- 21 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 1a0000 -- 23 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 1c0000 -- 25 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 1e0000 -- 26 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 200000 -- 28 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 220000 -- 30 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 240000 -- 32 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 260000 -- 33 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 280000 -- 35 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 2a0000 -- 37 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 2c0000 -- 39 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 2e0000 -- 41 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 300000 -- 42 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 320000 -- 44 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 340000 -- 46 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 360000 -- 48 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 380000 -- 50 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 3a0000 -- 51 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 3c0000 -- 53 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 3e0000 -- 55 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 400000 -- 57 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 420000 -- 58 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 440000 -- 60 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 460000 -- 62 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 480000 -- 64 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 4a0000 -- 66 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 4c0000 -- 67 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 4e0000 -- 69 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 500000 -- 71 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 520000 -- 73 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 540000 -- 75 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 560000 -- 76 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 580000 -- 78 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 5a0000 -- 80 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 5c0000 -- 82 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 5e0000 -- 83 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 600000 -- 85 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 620000 -- 87 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 640000 -- 89 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 660000 -- 91 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 680000 -- 92 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 6a0000 -- 94 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 6c0000 -- 96 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 6e0000 -- 98 % complete flash_erase: Cleanmarker Updated.
Erasing 128 Kibyte @ 6e0000 -- 100 % complete

Create a mount point for the partition:

root@R7800RT1:~# mkdir /jffs

Mount the partition block device:

root@R7800RT1:~# mount -t jffs2 /dev/mtdblock7 /jffs

Display size of /jffs:

root@R7800RT1:~# df -h /jffs

Filesystem                Size      Used Available Use% Mounted on
/dev/mtdblock7            7.0M    516.0K      6.5M   7% /jffs

Create and store a file on /jffs:

root@R7800RT1:~# echo 'Hello World!!!' > /jffs/helloworld.txt

root@R7800RT1:~# ls -l /jffs
-rw-r--r-- 1 root root 15 Jul 3 04:18 helloworld.txt

root@R7800RT1:~# cat /jffs/helloworld.txt
Hello World!!!

Copy the file to root's directory (/root) and preserve attributes:

root@R7800RT1:~# cp -a /jffs/helloworld.txt .

root@R7800RT1:~# ls -l
-rw-r--r-- 1 root root 15 Jul 3 04:18 helloworld.txt

Make the following changes to the original qcom-ipq8065-r7800.dts:

                                        jffs@7900000 {
						label = "jffs";
						reg = <0x7900000 0x0700000>;
						/* read-only;
                                                 * experimental 
                                                 * mtd7 
                                                 */ 
	     			        };

This was a lot of work for a measly 7MB. May not be worth doing!!

1282

Updated to latest 4.14.53. Router to client still capped below 20Mbps. Both VPN and samba capped at 20Mbps. Anyone?

1283

I am experiencing the same issue with a build from yesterday. Very odd, the cpus aren't loaded heavily or anything. Going back to an earlier build with the same settings solves the problem.

edit: new build this morning seems to be working for me.

1284

I tried @hnyman build and the 20Mbps capped issue gone. Probably my build environment which is Arch Linux with some latest package shit.

1285

Has anyone tried kvalo's latest QCA9984 firmware-5.bin_10.4-3.6-00144 ?

Fast firmware upgrade process:

root@R7800RT1:~# cd /lib/firmware/ath10k/QCA9984/hw1.0

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# ls
board-2.bin firmware-5.bin

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# mv firmware-5.bin firmware-5.bin.53

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# wget https://github.com/kvalo/ath10k-firmware/raw/master/QCA9984/hw1.0/3.6/firmware-5.bin_10.4-3.6-00144

--2018-07-07 19:57:57--  https://github.com/kvalo/ath10k-firmware/raw/master/QCA9984/hw1.0/3.6/firmware-5.bin_10.4-3.6-00144
Resolving github.com... 192.30.253.112, 192.30.253.113, 64:ff9b::c01e:fd70, ...
Connecting to github.com|192.30.253.112|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://raw.githubusercontent.com/kvalo/ath10k-firmware/master/QCA9984/hw1.0/3.6/firmware-5.bin_10.4-3.6-00144 [following]
--2018-07-07 19:57:58--  https://raw.githubusercontent.com/kvalo/ath10k-firmware/master/QCA9984/hw1.0/3.6/firmware-5.bin_10.4-3.6-00144
Resolving raw.githubusercontent.com... 151.101.56.133, 64:ff9b::9765:3885
Connecting to raw.githubusercontent.com|151.101.56.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 661008 (646K) [application/octet-stream]
Saving to: 'firmware-5.bin_10.4-3.6-00144'

firmware-5.bin_10.4-3.6-00144  100%[=================================================>] 645.52K   425KB/s    in 1.5s    

2018-07-07 19:58:00 (425 KB/s) - 'firmware-5.bin_10.4-3.6-00144' saved [661008/661008]

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# ls
board-2.bin firmware-5.bin.53 firmware-5.bin_10.4-3.6-00144

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# mv firmware-5.bin_10.4-3.6-00144 firmware-5.bin

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# ls
board-2.bin firmware-5.bin firmware-5.bin.53

root@R7800RT1:/lib/firmware/ath10k/QCA9984/hw1.0# reboot

root@R7800RT1:~# dmesg|grep 'firmware ver'
[ 18.336229] ath10k_pci 0000:01:00.0: firmware ver 10.4-3.6-00144 api 5 features no-p2p,mfp,peer-flow-ctrl,btcoex-param,allows-mesh-bcast,no-ps crc32 e385d6b0
[ 26.954228] ath10k_pci 0001:01:00.0: firmware ver 10.4-3.6-00144 api 5 features no-p2p,mfp,peer-flow-ctrl,btcoex-param,allows-mesh-bcast,no-ps crc32 e385d6b0

For me, the performance is about the same.

1286

Why would it be different?

1287

DSA patch rebased to current trunk: https://gist.github.com/neheb/011cad5943ed43c8cddf43df7388e189

1 Like
1288

can you check about the switch with dsa ? any improvement ?

1289

The latency issues with the swconfig driver are non-existent with DSA. Not sure what you're looking for.

1290

with dsa the driver should change with the kernel mainline...

I remember that there were some problem with it...