[PR] ipq806x: kernel 5.10 bump code propose

I made Test-DSA-kernel510-master-r16313-851dadc257-20210323 for my R7800 community build.

 OpenWrt SNAPSHOT, r16313-851dadc257
 -----------------------------------------------------
root@router1:~# uname -a
Linux router1 5.10.23 #0 SMP Tue Mar 23 17:07:31 2021 armv7l GNU/Linux

root@router1:~# cat /sys/kernel/debug/regulator/regulator_summary
 regulator                      use open bypass  opmode voltage current     min     max
---------------------------------------------------------------------------------------
 regulator-dummy                 13   13      0 unknown     0mV     0mA     0mV     0mV
    29000000.sata-target          1                                 0mA     0mV     0mV
    29000000.sata-phy             1                                 0mA     0mV     0mV
    29000000.sata-ahci            1                                 0mA     0mV     0mV
    1b700000.pci-vdda_refclk      1                                 0mA     0mV     0mV
    1b700000.pci-vdda_phy         1                                 0mA     0mV     0mV
    1b700000.pci-vdda             1                                 0mA     0mV     0mV
    1b500000.pci-vdda_refclk      1                                 0mA     0mV     0mV
    1b500000.pci-vdda_phy         1                                 0mA     0mV     0mV
    1b500000.pci-vdda             1                                 0mA     0mV     0mV
    s1a                           1    1      0 unknown  1100mV     0mA  1050mV  1150mV
       soc:l2-cache-l2            1                                 0mA  1100mV  1100mV
    s1b                           0    0      0 unknown  1050mV     0mA  1050mV  1150mV
    s2a                           1    1      0 unknown   837mV     0mA   775mV  1275mV
       cpu0-cpu                   1                                 0mA   831mV   918mV
    s2b                           1    1      0 unknown   837mV     0mA   775mV  1275mV
       cpu1-cpu                   1                                 0mA   831mV   918mV
 SDCC Power                       1    0      0 unknown  3300mV     0mA  3300mV  3300mV

During flent speedtest:

root@router1:/tmp/rrd/router1/ping# cat /sys/kernel/debug/regulator/regulator_summary
 regulator                      use open bypass  opmode voltage current     min     max
---------------------------------------------------------------------------------------
 regulator-dummy                 13   13      0 unknown     0mV     0mA     0mV     0mV
    29000000.sata-target          1                                 0mA     0mV     0mV
    29000000.sata-phy             1                                 0mA     0mV     0mV
    29000000.sata-ahci            1                                 0mA     0mV     0mV
    1b700000.pci-vdda_refclk      1                                 0mA     0mV     0mV
    1b700000.pci-vdda_phy         1                                 0mA     0mV     0mV
    1b700000.pci-vdda             1                                 0mA     0mV     0mV
    1b500000.pci-vdda_refclk      1                                 0mA     0mV     0mV
    1b500000.pci-vdda_phy         1                                 0mA     0mV     0mV
    1b500000.pci-vdda             1                                 0mA     0mV     0mV
    s1a                           1    1      0 unknown  1100mV     0mA  1050mV  1150mV
       soc:l2-cache-l2            1                                 0mA  1100mV  1100mV
    s1b                           0    0      0 unknown  1050mV     0mA  1050mV  1150mV
    s2a                           1    1      0 unknown  1025mV     0mA   775mV  1275mV
       cpu0-cpu                   1                                 0mA  1021mV  1128mV
    s2b                           1    1      0 unknown   837mV     0mA   775mV  1275mV
       cpu1-cpu                   1                                 0mA   831mV   918mV
 SDCC Power                       1    0      0 unknown  3300mV     0mA  3300mV  3300mV

Ps. great that you rebased your branch.

1 Like

@Ansuel I tested @hnyman's build Test-DSA-kernel510-master-r16313-851dadc257-20210323 and my R7800 booted successfully.

However I was not able to get VLAN tagged traffic from/to my Raspberry Pi 4B (main router, running 21.02-SNAPSHOT) working on the DSA build. I am not sure if I am doing something wrong in the config or is it a bug. I reverted back to 21.02-SNAPSHOT swconfig build for now. I will try DSA again maybe during the weekend. Glad I could help in debugging the voltage issue. Thank you for your work on DSA.

@hnyman Thank you for the build.

This is the default config of a realtek/ rtl838x device (gs1900-8), using DSA to configure the management VLAN (well, the lan network) on port 1, tagged with VID100:

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fdXX:XXXX:XXXX::/48'

config device 'switch'
	option name 'switch'
	option type 'bridge'
	option macaddr 'fc:f5:28:XX:XX:Xe'

config bridge-vlan 'wan_vlan'
	option device 'switch'
	option vlan '1'
	option ports ' lan1 lan2 lan3 lan4 lan5 lan6 lan7 lan8'

config interface 'wan'
	option ifname 'switch.1'
	option proto 'dhcp'

config device 'wan_switch_1_dev'
	option name 'switch.1'
	option macaddr 'fc:f5:28:XX:XX:Xe'

config interface 'wan6'
	option ifname 'switch.1'
	option proto 'dhcpv6'

config bridge-vlan 'lan_vlan'
	option device 'switch'
	option vlan '100'
	option ports 'lan1:t'

config interface 'lan'
	option ifname 'switch.100'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'
	option ip6assign '60'

config device 'lan_switch_100_dev'
	option name 'switch.100'
	option macaddr 'fe:f5:28:XX:XX:Xe'

Thanks @slh. I got DSA config working on my R7800 that I am using as a Ethernet Switch and WiFi access Point

My R7800 setup is

WAN port: Modem
LAN1 - Raspberry Pi 4 Model B - Main Router running OpenWRT 21.02-SNAPSHOT
LAN2 - Nothing
LAN3 - lan_2 (guest lan network)
LAN4 - lan_1 aka main local network
2.4 GHz WiFi - lan_2 guest network
5 GHz WiFi - lan_1 local network

I am using VLAN tagged traffic on LAN1 port of R7800 to separate "internet" (VLAN ID 1001), "lan_1" (VLAN ID 2001) and "lan_2" (VLAN ID 2002) packets from/to my Raspberry Pi 4B.

My R7800 DSA /etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd00:0000:0000::/48'
	option packet_steering '1'

config interface 'lan_0'
	option proto 'static'
	option defaultroute '0'
	option delegate '0'
	list ipaddr '192.168.1.200/24'
	option ifname 'switch0.2001'

config interface 'wan'
	option proto 'none'
	option force_link '1'
	option defaultroute '0'
	option peerdns '0'
	option delegate '0'
	option ifname 'switch0.1001'

config interface 'lan_1_dhcpv4'
	option proto 'dhcp'
	option delegate '0'
	option ifname 'switch0.2001'

config interface 'lan_1_dhcpv6'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'no'
	option delegate '0'
	option ifname 'switch0.2001'

config interface 'lan_2'
	option proto 'none'
	option defaultroute '0'
	option peerdns '0'
	option delegate '0'
	option ifname 'switch0.2002'
	option force_link '1'

config bridge-vlan
	option device 'switch0'
	option vlan '1001'
	list ports 'lan1:t'
	list ports 'wan'

config bridge-vlan
	option device 'switch0'
	option vlan '2001'
	list ports 'lan1:t'
	list ports 'lan2'
	list ports 'lan4'

config bridge-vlan
	option device 'switch0'
	option vlan '2002'
	list ports 'lan1:t'
	list ports 'lan3'

config device
	option type 'bridge'
	option name 'switch0'
	list ifname 'lan1'
	list ifname 'lan2'
	list ifname 'lan3'
	list ifname 'lan4'
	list ifname 'wan'

config device
	option name 'switch0.2001'
	option acceptlocal '1'

@hnyman what changes did you do to the gov on your firmware ?

I have hast modified the ondemand governor to be be somewhat more aggressive:

--- /dev/null
+++ b/target/linux/ipq806x/patches-5.10/910-tweak-cpufreq-ondemand-governor.patch
@@ -0,0 +1,16 @@
+--- a/drivers/cpufreq/cpufreq_ondemand.c
++++ b/drivers/cpufreq/cpufreq_ondemand.c
+@@ -18,10 +18,10 @@
+ #include "cpufreq_ondemand.h"
+ 
+ /* On-demand governor macros */
+-#define DEF_FREQUENCY_UP_THRESHOLD		(80)
+-#define DEF_SAMPLING_DOWN_FACTOR		(1)
++#define DEF_FREQUENCY_UP_THRESHOLD		(60)
++#define DEF_SAMPLING_DOWN_FACTOR		(10)
+ #define MAX_SAMPLING_DOWN_FACTOR		(100000)
+-#define MICRO_FREQUENCY_UP_THRESHOLD		(95)
++#define MICRO_FREQUENCY_UP_THRESHOLD		(60)
+ #define MICRO_FREQUENCY_MIN_SAMPLE_RATE		(10000)
+ #define MIN_FREQUENCY_UP_THRESHOLD		(1)
+ #define MAX_FREQUENCY_UP_THRESHOLD		(100)

But the the 384k step still have visible impact in flent testing.
This is from flent. I tested ondemand governor with the default 384k step enabled, then 600k set as the scaling_min_freq, and finally with performance governor. There is not much difference between performance and the 600k min, but the 384k min lets lots of bandwidth unused. SQM is used.

1 Like

The original firmware set min Freq with 800mhz... So your solution would be remove the 300mhz opp or set the min freq in the gov. Think it's cleaner to set the min Freq with a startup script by default so that if someone wants to keep the low settings, he can change that.
Also nice graph. Wonder if openwrt can create a custom startup script for this. (A boot init.d like it's done for mvebu to reset the boot counter?)

2 Likes

That is actually a nice idea (at least for my own build). I tried having it a script setting scaling_min_freq in /etc/rc.local (but that got lost in some reset).

However, I am more worried about R7800 users in general, so a global startup modification might be viable.

Flent is pretty cool tool for network performance measurement :wink: A bit more comprehensive that a casual speed test.

Those three test runs were run from the same PC inside a 10 minutes range, so the only real change was on the CPU frequency scaling settings. So the impact from the scaling is quite visible.

+1 vote for setting min freq higher (up to 800MHz) but from different perspective.

At least late last year there were issues causing spontaneous restarts and pointing to Krait CPU freq changes.

https://bugs.openwrt.org/index.php?do=details&task_id=3099

I observed the same spontaneous restart in last Nov, and while the workaround mentioned in the bug report was to use performance governor, I still use the ondemand and instead set the min freq to 800MHz in rc.local.

Have not experienced any restarts due to same issue after changing min freq and the freq changes has drastically reduced after;

1 Like

Would be good to understand if 600mgz can also be used just to reduce power consumption even more. (Very old test were not right as the voltage was actually never changed and ethernet EEE was disabled)

Anyway I think I will propose a startup script that tweak the governor as removing the 300 opp Freq from the dts seems too aggressive and would remove user choise.

(Wonder if the nss build crash were caused by the CPU scaling and not the nss core scaling)

1 Like

Highly possible as it is very difficult to get logs for spontaneous restarts.
But would not tie up the reported CPU freq changes bug report to nss build as it was reported by non nss build user.

Could be that the problem is more present since you stress more the chip using the nss core and the regulator since you use more power...

Thanks for the idea.
I have so far done the tuning by patching kernel before compilation, as I saw /etc/rc.local as too vulnerable. But your idea about /etc/init.d/something is great. It makes the frequency scaling more easily configurable by the user, but is still not as vulnerable as a config setting or user script.

This seems to work ok for me in R7800 (combined with the normal symlink in /etc/rc.d).

root@router1:~# cat /etc/init.d/cpufreq
#!/bin/sh /etc/rc.common

START=15

boot() {
  # Select 'ondemand'=scaling or 'performance'=always max freq
  echo ondemand > /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
  echo ondemand > /sys/devices/system/cpu/cpufreq/policy1/scaling_governor

  # Effective only with ondemand
  echo 600000 > /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq
  echo 600000 > /sys/devices/system/cpu/cpufreq/policy1/scaling_min_freq
  echo 10 > /sys/devices/system/cpu/cpufreq/ondemand/sampling_down_factor
  echo 50 > /sys/devices/system/cpu/cpufreq/ondemand/up_threshold
}
1 Like

Joined the bandwagon and compiled Ansuel ipq806x-5.10 on top of the latest master today. TFTP factory img + manual config, no sysupgrade.

Just for the sake of it run dslreports speed test and it is still quite a bit behind current swconfig@master where I get 600-700Mbps both up/down. No SQM or SW offloading used. Test run with performance governor and /etc/config/network config is standard "factory".

image

Sirq hit max 50%, assuming loading only the CPU-0.

Otherwise the logs are clean so far and seems to be working ok.

Edit:
br-lan seems to be going up and down pretty frequently;

Sat Mar 27 20:38:11 2021 kern.info kernel: [ 5831.161832] qca8k 37000000.mdio-mii:10 lan4: Link is Down
Sat Mar 27 20:38:11 2021 kern.info kernel: [ 5831.161946] br-lan: port 4(lan4) entered disabled state
Sat Mar 27 20:38:14 2021 kern.info kernel: [ 5834.283224] qca8k 37000000.mdio-mii:10 lan4: Link is Up - 1Gbps/Full - flow control rx/tx
Sat Mar 27 20:38:14 2021 kern.info kernel: [ 5834.283292] br-lan: port 4(lan4) entered blocking state
Sat Mar 27 20:38:14 2021 kern.info kernel: [ 5834.290473] br-lan: port 4(lan4) entered forwarding state
Sat Mar 27 20:38:14 2021 daemon.notice netifd: Network device 'lan4' link is up

Edit2:
Based on earlier reported issue on lan going down/up, disabled eee with ethtool --set-eee lan4 eee off.
Let's see if any help.

1 Like

DSA only uses one CPU for now.

Just a quick update, I got an image to boot and run:

[    0.000000] Booting Linux on physical CPU 0x0
[    0.000000] Linux version 5.10.26 (sn@DESKTOP-TPH49OG) (arm-openwrt-linux-muslgnueabi-gcc (OpenWrt GCC 8.4.0 r16313+21-851dadc257b7) 8.4.0, GNU ld (GNU Binutils) 2.34) #0 SMP Fri Mar 26 20:43:29 2021
[    0.000000] CPU: ARMv7 Processor [512f04d0] revision 0 (ARMv7), cr=10c5787d
[    0.000000] CPU: div instructions available: patching division code
[    0.000000] CPU: PIPT / VIPT nonaliasing data cache, PIPT instruction cache
[    0.000000] OF: fdt: Machine model: Netgear Nighthawk X4 R7500v2

After a few attempts configuring DSA (one of which made my device fail to finish booting - it's fragile to mistakes) i did get my AP only, 2 VLAN network up but it was not stable wrt to ipv4. I don't know why yet as my testing window closed when my family started asking for internet (I did not get a chance to disable eee as described above).

I can post a gist with a full dmesg, regulator_summary, plus some other system info (temperatures and cpuidle seem to be working) if you like. I'll see if i can test again tomorrow and try disabling eee to keep the network stable for longer.

Note I still get the build error I mention above. First time through after make clean/distclean no images are generated but after doing /usr/bin/make -j 1 IGNORE_ERRORS=m V=s immediatly after the build failure, images are generated (which i used for the test) but the same compile error still comes up.

I don't know why this happens yet, it looks like the command that generates qcom-ipq8064-g10.dtb silently fails.

HTH

are you sure the swconfig results are not with sw offload enabled?
Anyway i wonder if using only one cpu is the cause of the port dropping

Just changed back to today master build to run the same dslreports test:

root@R7800:~# ubus call system board
{
	"kernel": "5.4.106",
	"hostname": "R7800",
	"system": "ARMv7 Processor rev 0 (v7l)",
	"model": "Netgear Nighthawk X4S R7800",
	"board_name": "netgear,r7800",
	"release": {
		"distribution": "OpenWrt",
		"version": "SNAPSHOT",
		"revision": "r16345-d71424a085",
		"target": "ipq806x/generic",
		"description": "OpenWrt SNAPSHOT r16345-d71424a085"
	}
}
root@R7800:~#

Dslreports run with standard ondemand governor with min freq 800MHz:

root@R7800:~# cat /sys/devices/system/cpu/cpufreq/policy0/scaling_governor
ondemand
root@R7800:~# cat /sys/devices/system/cpu/cpufreq/policy1/scaling_governor
ondemand
root@R7800:~# cat /sys/devices/system/cpu/cpufreq/policy0/scaling_min_freq
800000
root@R7800:~# cat /sys/devices/system/cpu/cpufreq/policy1/scaling_min_freq
800000

image

And at least I've not tick the box for SW offloading;

Edit:
That port dropping can be related to the same eee related issue @rog reported earlier.
I've now disabled eee for all the lan & wan ports.
So need to wait and see if I'll get those port drops anymore.

2 Likes

I tried again; however, this time turning off eee on lan1, lan2, lan3, lan4, and wan.

So far then network and device are stable so I can let this go for at least today. I'll post a link to the dmesg, a regulator_smmary, /etc/config/network (for dsa) plus some other configuration output in a bit.

I tried to set stp and igmp snooping via luci for the one bridge defined and lost connectivity to the device. I had to use my serial cable to access the device, theses options were not enabled but I am able to set these options and restart the network through the command line without issue. I'll report this in the luci DSA config thread as well and post logs.

I'm in and out today, so please be patient for the logs...

1 Like

Anyway i'm splitting the 5.10 pr from dsa since it still not ready and require more testing. In theory this should make the merge of the 5.10 commits quicker and less patch will be needed to test dsa

1 Like