I have a question regarding 19.07 stable.
Linux R7800 4.14.221 #0 SMP Mon Feb 22 15:36:55 2021 armv7l GNU/Linux
Recently I looked into the /proc/interrupts and figured out that interrupts are completely imbalanced between CPU cores.
All IRQ processing is performed on CPU0 but at the same time I can observe a near-zero IRQ load on CPU1.
It seems like the packet steering (RPS/XPS) is disabled somewhere globally in your build.
Would you please to advise me how to workaround this issue.
Thank you in advance!
First of all I have verified that RPS/XPS queue processing policies are configured already for eth0/eth1 devices (in a default setup).
The dump below shows that RPS/XPS queues for eth0/eth1 are already set up to be processed on CPU1 (decimal 2 = bitmask 10).
root@R7800:~# grep . /sys/class/net/eth?/queues/?x-0/?ps_cpus
/sys/class/net/eth0/queues/rx-0/rps_cpus: 2
/sys/class/net/eth0/queues/tx-0/xps_cpus: 2
/sys/class/net/eth1/queues/rx-0/rps_cpus: 2
/sys/class/net/eth1/queues/tx-0/xps_cpus: 2
But the /proc/interrupts dump shows zero IRQs from eth0/eth1 were processed by CPU1.
root@R7800:~# cat /proc/interrupts
CPU0 CPU1
16: 27251761 10341481 GIC-0 18 Edge gp_timer
18: 33 0 GIC-0 51 Edge qcom_rpm_ack
19: 0 0 GIC-0 53 Edge qcom_rpm_err
20: 0 0 GIC-0 54 Edge qcom_rpm_wakeup
26: 0 0 GIC-0 241 Edge ahci[29000000.sata]
27: 0 0 GIC-0 210 Edge tsens_interrupt
28: 329028 0 GIC-0 67 Edge qcom-pcie-msi
29: 33 0 GIC-0 89 Edge qcom-pcie-msi
30: 464664 0 GIC-0 202 Edge adm_dma
31: 44722532 0 GIC-0 255 Level eth0
32: 38133156 0 GIC-0 258 Level eth1
33: 0 0 GIC-0 130 Level bam_dma
34: 0 0 GIC-0 128 Level bam_dma
35: 0 0 PCI-MSI 0 Edge aerdrv
36: 329028 0 PCI-MSI 1 Edge ath10k_pci
68: 0 0 PCI-MSI 0 Edge aerdrv
69: 33 0 PCI-MSI 1 Edge ath10k_pci
101: 13 0 GIC-0 184 Level msm_serial0
102: 2 0 msmgpio 6 Edge keys
103: 2 0 msmgpio 54 Edge keys
104: 2 0 msmgpio 65 Edge keys
105: 0 0 GIC-0 142 Level xhci-hcd:usb1
106: 0 0 GIC-0 237 Level xhci-hcd:usb3
IPI0: 0 0 CPU wakeup interrupts
IPI1: 0 0 Timer broadcast interrupts
IPI2: 7575209 4041545 Rescheduling interrupts
IPI3: 26798724 75092959 Function call interrupts
IPI4: 0 0 CPU stop interrupts
IPI5: 9195359 3254587 IRQ work interrupts
IPI6: 0 0 completion interrupts
Err: 0
Note: All results above have been gathered without any IRQ tweaks or custom balancing.
The next thing I have tried was to change RPS/XPS policy to instruct kernel that BOTH CPU cores should process packets from any network device queue in the system including eth0/eth1 queues (by setting bitmask to decimal 3 = binary '11').
Unfortunately I got absolutely the same result as above - no packets have been processed by CPU1.
for file in /sys/class/net/* ; do
echo 3 > "${file}/queues/rx-0/rps_cpus"
echo 3 > "${file}/queues/tx-0/xps_cpus"
done
PS: My purpose is to redistribute RPS/XPS ethernet packet processing between cores prior to enabling the WiFi interface that will strictly bound to CPU0 (according information from this forum).
For now I see no need in using of irqbalance. I think that manual redistribution will be enough for my needs.
System info:
root@R7800:~# ubus call system board
{
"kernel": "4.14.221",
"hostname": "R7800",
"system": "ARMv7 Processor rev 0 (v7l)",
"model": "Netgear Nighthawk X4S R7800",
"board_name": "netgear,r7800",
"release": {
"distribution": "OpenWrt",
"version": "19.07-SNAPSHOT",
"revision": "r11312-e9c0c5021c",
"target": "ipq806x/generic",
"description": "OpenWrt 19.07-SNAPSHOT r11312-e9c0c5021c"
}
}