Packet steering vs irqbalance on WRT1900ACS v2

Hardware Questions and Recommendations
1

On my WRT1900ACS v2, I enabled packet-steering, ran some wan speed and buffer bloat tests, local iperf3 tests and then checked cat /proc/interrupts and saw almost no impact. But after i downloaded and enabled irqbalance, did the same tests and /proc/interrupts results showed one of the wifi devices (73) was now utilizing the second cpu (CPU1) exclusively instead of everything just being on CPU0. My question is, should I leave Packet steering enabled at the same time as irqbalance as I saw no change with PS enabled in the first place?

Result after enabling irqbalance: CPU1 is now looking to be EXCLUSIVELY used by mlwifi (73). Marked with **. The interrupt count on CPU0 of 7238082 stopped increasing after enabling irqbalance.

          CPU0       CPU1       
 17:          0          0     GIC-0  27 Edge      gt
 18:     948152     942882     GIC-0  29 Edge      twd
 19:          0          0      MPIC   5 Level     armada_370_xp_per_cpu_tick
 21:    1386080          0     GIC-0  34 Level     mv64xxx_i2c
 22:         20          0     GIC-0  44 Level     ttyS0
 32:          0          0     GIC-0  41 Level     f1020300.watchdog
 36:          0          0     GIC-0  96 Level     f1020300.watchdog
 37:    2337377          0      MPIC   8 Level     eth0
 38:      32104          0     GIC-0  50 Level     ehci_hcd:usb1
 39:          0          0     GIC-0  51 Level     f1090000.crypto
 40:          0          0     GIC-0  52 Level     f1090000.crypto
 41:          0          0     GIC-0  58 Level     ahci-mvebu[f10a8000.sata]
 42:         32          0     GIC-0 116 Level     marvell-nfc
 43:          0          0     GIC-0  49 Level     xhci-hcd:usb2
 44:          2          0     GIC-0  54 Level     f1060800.xor
 45:          2          0     GIC-0  97 Level     f1060900.xor
 49:          0          0  mv88e6xxx-g1   3 Edge      mv88e6xxx-g1-atu-prob
 51:          0          0  mv88e6xxx-g1   5 Edge      mv88e6xxx-g1-vtu-prob
 53:          5          3  mv88e6xxx-g1   7 Edge      mv88e6xxx-g2
 55:          2          1  mv88e6xxx-g2   0 Edge      mv88e6xxx-1:00
 56:          0          0  mv88e6xxx-g2   1 Edge      mv88e6xxx-1:01
 57:          0          0  mv88e6xxx-g2   2 Edge      mv88e6xxx-1:02
 58:          3          0  mv88e6xxx-g2   3 Edge      mv88e6xxx-1:03
 59:          0          2  mv88e6xxx-g2   4 Edge      mv88e6xxx-1:04
 70:          0          0  mv88e6xxx-g2  15 Edge      mv88e6xxx-watchdog
 71:          0          0  f1018100.gpio  24 Edge      gpio-keys
 72:          0          0  f1018100.gpio  29 Edge      gpio-keys
 **73:    7238082    8932373     GIC-0  61 Level     mwlwifi**
 74:   16323037          0     GIC-0  65 Level     mwlwifi
IPI0:          0          1  CPU wakeup interrupts
IPI1:          0          0  Timer broadcast interrupts
IPI2:     214350     835561  Rescheduling interrupts
IPI3:        921       1560  Function call interrupts
IPI4:          0          0  CPU stop interrupts
IPI5:          0          0  IRQ work interrupts
IPI6:          0          0  completion interrupts
Err:          0
1 Like
2

Whether packet steering helps really depends on the number of CPUs and the number of NICs. And which CPUs are actually configured for packet processing... on a dual core packet steering will only help if it does not exclude the CPU processing the NIC interrupts, as that means further processing happens only on the other CPU.

So, I think you need to test all combinations and see whether it helps for your load.

1 Like
3

That clarifies things a bit, thanks.

I ran a couple of different combinations of wifi and wired across multiple ports and vlans what I've noticed is that irqbalance just flips a switch so that one of the wifis utilizes CPU1 and thats it. It doesnt perform any sort of balancing act afterwards. I'm not complaining or anything as my cpu utilization has not peaked above 20-30%. Just trying to figure whether keeping both enabled at the same time might impact performance down the line as I dont have a full understanding of the inner workings of either.

4

On mvebu target probably want packet steering enabled.

1 Like
5

I think irqbalance can run in single shot mode or in a mode where it repeatedly runs to re-balance things, not sure what the OpenWrt default behavior is.

6

iirc you can peg affinity for mwlwifi IRQs and tell irqbalance to ignore:

	# List of IRQ's to ignore
	#list banirq '36'
	#list banirq '69'
7

That email thread is now almost 3 years old, we are now several major kernel versions newer and have a 22.03 release. Do we know if packet steering should still be enabled on mvebu?

8

afaik, nothing has changed.

9

This thread is actually from Feb 19 of this year (2022) not Feb '19. At first glace, it would seem that way. This was actually on discussion with OpenWRT v21.