And where is that?
So I can use htop and then change the init for it to be on the other core?
Where all normal uci config files are...
/etc/config/irqbalance
1 Like
htop doesn't really enter into it except that i used it to show the cpu usage and then screenshot it as an illustration....
To figure out what processes I need to change, is it not a valid strategy?
Sure, yeah - it's a very useful tool for that
Could you also share how you distributed the IRQs for the ixgbe NIC driver, to spread them across cores 1-6?
This is the setting that yields the result shown in the htop screenshot above
This is very specific to the NIC. If you don't have the same NICs as me, this won't help you at all
<interface> <cpu core> <hex mask> <interrupt>
eth0 - my wan interface
eth0 1 0x02 -> /proc/irq/47/smp_affinity
eth0 2 0x04 -> /proc/irq/48/smp_affinity
eth0 3 0x08 -> /proc/irq/49/smp_affinity
eth0 4 0x10 -> /proc/irq/50/smp_affinity
eth0 5 0x20 -> /proc/irq/51/smp_affinity
eth0 6 0x40 -> /proc/irq/52/smp_affinity
eth0 1 0x02 -> /proc/irq/53/smp_affinity
eth0 2 0x04 -> /proc/irq/54/smp_affinity
eth3 - my lan interface
eth3 0 0x01 -> /proc/irq/74/smp_affinity
eth3 1 0x02 -> /proc/irq/75/smp_affinity
eth3 2 0x04 -> /proc/irq/76/smp_affinity
eth3 3 0x08 -> /proc/irq/77/smp_affinity
eth3 4 0x10 -> /proc/irq/78/smp_affinity
eth3 5 0x20 -> /proc/irq/79/smp_affinity
eth3 6 0x40 -> /proc/irq/80/smp_affinity
eth3 7 0x80 -> /proc/irq/81/smp_affinity
1 Like
And to contrast this, the following settings yield the following core load with a 1Gbps download stream
<interface> <cpu core> <hex mask> <interrupt>
eth0 - my wan interface
eth0 0 0x01 -> /proc/irq/47/smp_affinity
eth0 1 0x02 -> /proc/irq/48/smp_affinity
eth0 2 0x04 -> /proc/irq/49/smp_affinity
eth0 3 0x08 -> /proc/irq/50/smp_affinity
eth0 4 0x10 -> /proc/irq/51/smp_affinity
eth0 5 0x20 -> /proc/irq/52/smp_affinity
eth0 6 0x40 -> /proc/irq/53/smp_affinity
eth0 7 0x80 -> /proc/irq/54/smp_affinity
eth3 - my lan interface
eth3 0 0x01 -> /proc/irq/74/smp_affinity
eth3 1 0x02 -> /proc/irq/75/smp_affinity
eth3 2 0x04 -> /proc/irq/76/smp_affinity
eth3 3 0x08 -> /proc/irq/77/smp_affinity
eth3 4 0x10 -> /proc/irq/78/smp_affinity
eth3 5 0x20 -> /proc/irq/79/smp_affinity
eth3 6 0x40 -> /proc/irq/80/smp_affinity
eth3 7 0x80 -> /proc/irq/81/smp_affinity
And these ones yield this result
<interface> <cpu core> <hex mask> <interrupt>
eth0 - my wan interface
eth0 1 0x02 -> /proc/irq/47/smp_affinity
eth0 2 0x04 -> /proc/irq/48/smp_affinity
eth0 3 0x08 -> /proc/irq/49/smp_affinity
eth0 4 0x10 -> /proc/irq/50/smp_affinity
eth0 5 0x20 -> /proc/irq/51/smp_affinity
eth0 6 0x40 -> /proc/irq/52/smp_affinity
eth0 1 0x02 -> /proc/irq/53/smp_affinity
eth0 2 0x04 -> /proc/irq/54/smp_affinity
eth3 - my lan interface
eth3 1 0x02 -> /proc/irq/74/smp_affinity
eth3 2 0x04 -> /proc/irq/75/smp_affinity
eth3 3 0x08 -> /proc/irq/76/smp_affinity
eth3 4 0x10 -> /proc/irq/77/smp_affinity
eth3 5 0x20 -> /proc/irq/78/smp_affinity
eth3 6 0x40 -> /proc/irq/79/smp_affinity
eth3 1 0x02 -> /proc/irq/80/smp_affinity
eth3 2 0x04 -> /proc/irq/81/smp_affinity
1 Like
And this is 8 download and upload streams all with dscp marks on them corresponding to one of SQM's diffserv4 buckets, so exercising the cpu more heavily using the first optimized set of affinity masks I posted above
So, IMHO, manual affinity configuration is always superior to irqbalance, as even a slight difference can result in a huge change to the way your cores are utilized in a multi-core system.
And it's all very specific to the NIC. It just requires a bit of experimentation to find the best set of affinity masks for your hardware.
1 Like
I can't use that. seems like the ICU for core 2 on xrx200 is non existent or not used..
CPU0 CPU1
7: 373508 365968 MIPS 7 timer
8: 5757 4202 MIPS 0 IPI call
9: 30815 231492 MIPS 1 IPI resched
22: 149245 0 icu 22 spi_rx
23: 52631 0 icu 23 spi_tx
24: 0 0 icu 24 spi_err
62: 0 0 icu 62 1e101000.usb, dwc2_hsotg:usb1
63: 61322 0 icu 63 mei_cpe
72: 1243041 0 icu 72 xrx200_net_rx
73: 2332056 0 icu 73 xrx200_net_tx
91: 0 0 icu 91 1e106000.usb, dwc2_hsotg:usb2
96: 3648074 0 icu 96 ptm_mailbox_isr
112: 300 0 icu 112 asc_tx
113: 0 0 icu 113 asc_rx
114: 0 0 icu 114 asc_err
126: 0 0 icu 126 gptu
127: 0 0 icu 127 gptu
128: 0 0 icu 128 gptu
129: 0 0 icu 129 gptu
130: 0 0 icu 130 gptu
131: 0 0 icu 131 gptu
144: 0 0 icu 144 ath9k
161: 0 0 icu 161 ifx_pcie_rc0
All traffic goes though core 0 for some reason,will investigate, reply if you have ideas.
It does not look like it's unused. It looks like its affinity is set to core 0. Try
echo 0x02 > /proc/irq/73/smp_affinity
What does it do? This should set the affinity for net_tx to cpu 1
Crashes Ethernet communications and does not work, I needed to change 0x02 to 1 cause it gave me ash: write error: Invalid argument. ICE even under irqbalance does not get touched
hello how make 50/50 to dual core belkin rt3200
thanks
Does anyone know how to solve this irqbalance error? irqbalance is saying that is is enabled but stopped. syslog states daemon.warn /usr/sbin/irqbalance: Daemon couldn't be bound to the file-based socket.
I tried mkdir -p /run and service irqbalance restart but it is still stopped. Is irqbalance really worth it? I was testing my wireguard connection (300mbps) to my raspberry pi 4 and saw a hrtimer took XXXns message so thought about irqbalance to help.
Thanks for any advice
I believe irqbalance runs, analyzes the IRQs, rebalances them, and then exits. So it's normal. It doesn't run continuously
1 Like
Default is:
4687 root 1048 S /usr/sbin/irqbalance -f -t 10
so sleep for 10 seconds and resample.
That sounds like the non-default oneshot mode.
Normally irqbalance stays active and periodically rebalances things.
I thought the oneshot mode had become the default at some point. Maybe I'm wrong on that.
It looks like I am definitely wrong on that. I see that it is in fact running continuously on my RPi4 ... sorry for bad info!
i am also interested in this topic, on my belkin rt3200 (e8450) but i dont see at all how i could balance the load
I would like to do this because sqm with optical fiber the cpu on htop consumes up to 100% of its capacity and create BB unlike FQ codel
I thought I would use SQM only for the upload part with irq balance but I don't want to mess around,