Thats nice, thx!
But i have a question, if you are doing the switch to 5.15 instead of using it as a testing kernel, should not 5.10 be removed? like the patchs folder and such?
Everything working flawlessly!
Default values seem to be the other way around on mine:
root@OpenWrt:~# cat /sys/class/net/eth0/queues/rx-0/rps_cpus
00
root@OpenWrt:~# cat /sys/class/net/eth1/queues/rx-0/rps_cpus
00
root@OpenWrt:~# grep eth /proc/interrupts
35: 580672 0 0 0 0 0 GICv3 44 Level eth0
87: 0 1126184 0 0 0 0 ITS-MSI 524288 Edge eth1
root@OpenWrt:~# cat /proc/irq/35/smp_affinity
3f
root@OpenWrt:~# cat /proc/irq/87/smp_affinity
3f
Is there a PR open to have this done in official release?
You probably did something like packet steering or irqbalance or you downloaded another image from somewhere else(not an official image?) or its related to this:
Cause you can see on openwrt 22.03:
And even on master:
Nothing is changed, in other words, irq will be on cores 4 and 5 and the queues with rps won't spread cause they will be set to 00 after boot.
No it is the standard rc4 version from openwrt downloads…
Ok, but still i gave 3 more reasons to why yours is different right now.
Yeah but I did not touch anything, waited more than 10sec, did not install irqbalance etc
But anyway, my question was more to know if we can hope to have the right settings done in future official release or we will have to either manage it manually or through irqbalance?
I use R4S 4Gb only for 2 docker images (using as managed switch).
What is the best way to divide CPU load in this case?
Do I even need to divide it (I am not using R4S as a router)?
I was thinking something like this:
eth0 IRQ on A53 core - core0 --> I don't use eth0 (deleted wan/wan6)
eth0 queues on A53 core - core1 --> I don't use eth0 (deleted wan/wan6)
eth0 queues on A53 core - core2
eth1 IRQ on A53 core - core3
Docker1 (Unifi Controller) --> cores 0-2 (or 0-3 if I don't need IRQ on separate core)
Docker2 (HomeAssistant) --> cores 4-5
core0 --> docker1 (Unifi), eth0 IRQ (eth0 is not in use)
core1 --> docker1 (Unifi), eth0 queues (eth0 is not in use)
core2 --> docker1 (Unifi), eth1 queues
core3 --> eth1 IRQ (or also with Unifi)
core4 --> docker2 (HA)
core5 --> docker2 (HA)
docker1 (Unifi) - 1.2gb RAM
docker2 (HA) - 2gb RAM
Unifi - 2 AP, 10 devices (6 IoT, 2 phones, 2 notebooks)
HA - 6 IoT devices, 1 integration
My questions:
- Does dividing IRQ and queues make sense in my case?
- Is CPU dividing ok for these 2 containers (Unifi -> 0-2 or 0-3 and HA 4-5)?
- Is RAM dividing ok for these 2 containers (Unifi 1.2gb and HA 2gb)?
Has this changed in official builds? I'm running 22.03-SNAPSHOT r19509 (I built it yesterday), and rps_cpus is seems to be 3f by default now:
root@router:~# cat /sys/class/net/eth0/queues/rx-0/rps_cpus
3f
root@router:~# cat /sys/class/net/eth1/queues/rx-0/rps_cpus
3f
root@router:~#
BTW, just out of curiosity I've decided to compare the NanoPi R4S CPU performance with some other devices I have.
First I tried using openssl speed to do the benchmark, but the results were inconsistent and unreliable - I believe this might be due to the fact that openssl might use some sort of cryto acceleration depending on the SoC and algorithm.
So I decided to use coremark, which is available via opkg install. While it is a single-core benchmark, it gives a good insight about the relative CPU performance. I also installed and used taskset to benchmark both A53 and A72 NanoPi R4S cores. Below are the results of my benchmarks. It is possible to see that the A72 core has basically double the performance compared with the A53 cores.
PS: The Odroid C2 is running Odroid's Linux, while the other devices are running OpenWrt 22.03-SNAPSHOT r19509 (NanoPi) and r19512 (AX6S).
----------------------------------------------------------------------
Device SoC Cores Clock Coremark 1.0
----------------------------------------------------------------------
Odroid C2 Amlogic S905 4xA53 1.50 GHz 4997.917534
Redmi AX6S MediaTek MT7622BV 2xA53 1.35 GHz 4681.282671
NanoPI R4S Rockchip RK3399 4xA53+2xA72 1.4/1.8 GHz 4990.435000 (1)-A53
NanoPI R4S Rockchip RK3399 4xA53+2xA72 1.4/1.8 GHz 10535.742506 (2)-A72
----------------------------------------------------------------------
(1) A53 core: taskset 0x000000001 coremark
(2) A75 core: taskset 0x000000010 coremark
2 Likes
Idk about this "right settings" situation....
Cause you see, each person will have its own necessity = demanding different settings.
One thing is for sure, the current default for openwrt is enough to do gigabit speed for basic router stuff, but no one would buy a device like this just to do basic router stuff, so yeah default might not be enough and might need adjustments! So, i really don't know if a PR should be made to change anything or not.
Are you using default settings? Or did you change anything before checking those? Ext4 or squashfs?
Anyway i dont see any PR or code changes in openwrt github, that would indicate this behavior tbh.
It could be related to this like i said:
A device like the nanopi r4s as managed swtich? Which has only 2 ports? Why?
Why not install linux with docker and run those, instead of running them using openwrt?
Yes, default settings (of course I changed various configurations after a clean install but I not change anything related to CPU affinity).
I'm using squashfs.
Because I am more familiar with it. I need multiple VLANs and in Openwrt I know how to configure them (container works). I am also not sure how compatible each distro is with R4S and how much (more) resources will it need...
What linux distro would you recommend? Can you help me regarding CPU dividing in my previous post?
Ill flash a 22.03 snapshot on a testing sd card later and ill report back what i find.
1 Like
I stand corrected. I did change one configuration that could have affected this - I have enabled "Packet Steering" (but I have not installed irqbalance).
Sorry for missing this in my previous post.
Reading NanoPi R4S specs I had the understanding that the CPU clocks were supposed to be 1.5GHz for the little cores (A53) and 2.0GHz for the big cores (A72).
However, at least running the official OpenWrt build the clocks seems to be actually 1.4GHz (A53) and 1.8GHz (A72):
root@router:~# cat /sys/devices/system/cpu/cpu*/cpufreq/cpuinfo_max_freq
1416000
1416000
1416000
1416000
1800000
1800000
Is anyone running FriendlyElec just to check if the clocks are the same?
1 Like
That's the official specs by Rockchip
1 Like
@dsouza Like @diizzy said those are official specs for the rk3399, but friendlyelec lists the nano r4s which uses a rk3399 with 1.5 Ghz on the a53 cores and 2.0 Ghz on the a72 cores cause they did a little bit of overclock, in other words, if you use friendlywrt instead of openwrt you will get those clocks as promised by friendlyelec.
BTW some ppl push the rk3399 even higher with 2.2 GHz on the a72 cores and 1.8 GHz on the a53 cores like the chinese with immortalwrt, but than the vcore is high and some ppl say its dangerous whiles others don't!
You can safely use 1.6 Ghz on the a53 cores and 2.0 on the a72 cores and that is what i am using:
3 Likes