I have a usb SSD attached, transmission is using the disk 24/7 and samba4 occasionally. Do I need to install irqbalance? No?
This is a 20h result using @hnyman 's script
# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
9: 0 0 0 0 GIC-0 39 Level arch_mem_timer
13: 5408817 12560262 6089234 6028447 GIC-0 20 Level arch_timer
16: 2 0 0 0 GIC-0 354 Edge smp2p
17: 0 0 0 0 GIC-0 216 Level 4a9000.thermal-sensor
18: 0 0 0 0 GIC-0 239 Level bam_dma
21: 0 0 0 0 GIC-0 270 Level bam_dma
22: 4 0 0 0 GIC-0 340 Level msm_serial0
23: 148969 0 0 0 GIC-0 178 Level bam_dma
24: 0 0 0 0 GIC-0 35 Edge wdt_bark
25: 0 0 0 0 GIC-0 357 Edge q6v5 wdog
29: 5 0 0 0 GIC-0 348 Edge ce0
30: 1862749 0 0 0 GIC-0 347 Edge ce1
31: 32230 0 0 0 GIC-0 346 Edge ce2
32: 105688 0 0 0 GIC-0 343 Edge ce3
34: 0 0 0 0 GIC-0 443 Edge ce5
36: 1 0 0 0 GIC-0 72 Edge ce7
38: 0 0 0 0 GIC-0 334 Edge ce9
39: 0 0 0 0 GIC-0 333 Edge ce10
40: 0 0 0 0 GIC-0 69 Edge ce11
47: 0 0 0 0 GIC-0 323 Edge reo2ost-exception
48: 25903 0 0 0 GIC-0 322 Edge wbm2host-rx-release
49: 68 0 0 0 GIC-0 321 Edge reo2host-status
50: 0 0 0 109091 GIC-0 320 Edge reo2host-destination-ring4
51: 0 0 96983 0 GIC-0 271 Edge reo2host-destination-ring3
52: 0 89407 0 0 GIC-0 268 Edge reo2host-destination-ring2
53: 0 0 0 113326 GIC-0 267 Edge reo2host-destination-ring1
57: 1 0 0 0 GIC-0 263 Edge ppdu-end-interrupts-mac3
58: 0 0 0 0 GIC-0 262 Edge ppdu-end-interrupts-mac2
59: 1 0 0 620224 GIC-0 261 Edge ppdu-end-interrupts-mac1
60: 1 0 0 0 GIC-0 260 Edge rxdma2host-monitor-status-ring-mac3
61: 0 0 0 0 GIC-0 256 Edge rxdma2host-monitor-status-ring-mac2
62: 1 0 0 0 GIC-0 255 Edge rxdma2host-monitor-status-ring-mac1
63: 1 0 0 0 GIC-0 235 Edge host2rxdma-host-buf-ring-mac3
64: 0 0 0 0 GIC-0 215 Edge host2rxdma-host-buf-ring-mac2
65: 1 0 0 0 GIC-0 212 Edge host2rxdma-host-buf-ring-mac1
66: 0 0 0 0 GIC-0 211 Edge rxdma2host-destination-ring-mac3
67: 0 0 0 0 GIC-0 210 Edge rxdma2host-destination-ring-mac2
68: 0 0 0 0 GIC-0 209 Edge rxdma2host-destination-ring-mac1
73: 0 0 66211 0 GIC-0 191 Edge wbm2host-tx-completions-ring3
74: 0 121423 0 0 GIC-0 190 Edge wbm2host-tx-completions-ring2
75: 0 0 0 116932 GIC-0 189 Edge wbm2host-tx-completions-ring1
77: 35 0 0 0 GIC-0 47 Edge cpr3
78: 18319138 0 0 0 GIC-0 377 Level edma_txcmpl
79: 0 0 0 0 GIC-0 385 Level edma_rxfill
80: 13803650 0 0 0 GIC-0 393 Level edma_rxdesc
81: 0 0 0 0 GIC-0 376 Level edma_misc
82: 0 0 0 0 pmic_arb 51380237 Edge pm-adc5
83: 0 0 0 0 smp2p 0 Edge q6v5 fatal
84: 1 0 0 0 smp2p 1 Edge q6v5 ready
85: 0 0 0 0 smp2p 2 Edge q6v5 handover
86: 0 0 0 0 smp2p 3 Edge q6v5 stop
87: 0 0 0 0 msmgpio 34 Edge keys
88: 0 0 0 0 msmgpio 63 Edge keys
89: 1183828 0 0 0 GIC-0 172 Level xhci-hcd:usb1
90: 64 0 0 0 GIC-0 353 Edge glink-native
IPI0: 10332 9616 13842 14374 Rescheduling interrupts
IPI1: 2425221 5509045 5582703 5462254 Function call interrupts
IPI2: 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 CPU stop (for crash dump) interrupts
IPI4: 0 0 0 0 Timer broadcast interrupts
IPI5: 4645 4361 6784 4234 IRQ work interrupts
IPI6: 0 0 0 0 CPU wake-up interrupts
Err: 0
Well you cannot use the irqbalance package, it doesn't work or do anything.
But you seem to have been using the various scripts that statically assign CPU affinities to interrupts.
Speaking of that, I've been experiencing hangs during times of high traffic with the script I've posted yesterday. Sadly, nothing gets logged over remote logging and the unit needs a power-cycle to recover.
I'll try to configure netconsole to see if I can catch some log messages.
1 Like
Wirelessly Attended sysupgrade to SNAPSHOT r23104 yesterday!
Seems we don't need to stop wpad for sysupgrades from here on in?
It was pure luck
1 Like
I got hangs in traffic test when I had balanced the edma IRQs (that seem to handle lan/wan wired), so I restricted my script to just selected ath11k IRQs.
You might test with edma IRQs left at core0.
1 Like
Like: ?
case "$desc" in
arch*) ;;
ce*) ;;
xhci*) ;;
edma*) ;;
*) "$callback" "/proc/irq/$irq/smp_affinity" || ret=1 ;;
esac
I hope you have the serial to TTL cable handy 
1 Like
login as: root
root@172.16.17.1's password:
BusyBox v1.36.1 (2023-05-24 06:29:35 UTC) built-in shell (ash)
_______ ________ __
| |.-----.-----.-----.| | | |.----.| |_
| - || _ | -__| || | | || _|| _|
|_______|| __|_____|__|__||________||__| |____|
|__| W I R E L E S S F R E E D O M
-----------------------------------------------------
OpenWrt SNAPSHOT, r23104-ef98dc3b3e
-----------------------------------------------------
root@DL-WRX36:~# service irqbalance-manual status
active with no instances
Yes.
There is no running process (just as expected).
The init script from @fif simply sets the affinities and ends. There is no need for a permanent daemon process.
2 Likes
Has anyone tried with kernel 6.1 ?
Yes. These changes includes the feedback from @Spyros and @hnyman.
I hope it's not bad etiquette to repost the full thing to avoid confusion ...
Install this script in /etc/init.d/irqbalance-manual:
#!/bin/sh /etc/rc.common
# Version 2023-05-29
START=13
USE_PROCD=1
AFFINITY_MIN=2
AFFINITY_MAX=8
AFFINITY_ALL="$(printf %x $(( AFFINITY_MAX * 2 - 1 )))"
set_affinities() {
local callback="$1" irq desc ret=0
sed -nre 's!^[[:space:]]*([0-9]+):[[:space:]]+.*[[:space:]]GIC-0[[:space:]]+[0-9]+[[:space:]]+(Level|Edge)[[:space:]]+(.+)$!\1 \3! p' /proc/interrupts | \
while read irq desc
do
case "$desc" in
arch*) ;; # Properly balanced
ce*) ;; # Wifi firmware crashes
edma*) ;; # Hangs wifi on high throughput https://forum.openwrt.org/t/dynalink-dl-wrx36-askey-rt5010w-ipq8072a-technical-discussion/110454/1743
xhci*) ;; # Crashes with USB drive https://forum.openwrt.org/t/dynalink-dl-wrx36-askey-rt5010w-ipq8072a-technical-discussion/110454/1736
*) "$callback" "/proc/irq/$irq/smp_affinity" || ret=1 ;;
esac
done
return $ret
}
set_affinity_per_cpu() {
local procfile="$1" ret=0
echo "$AFFINITY" > "$procfile" || ret=$?
if [ $AFFINITY -ge $AFFINITY_MAX ]
then
AFFINITY=$AFFINITY_MIN
else
AFFINITY=$(( AFFINITY * 2 ))
fi
return $ret
}
set_affinity_shared() {
local procfile="$1" ret=0
echo "$AFFINITY_ALL" > "$procfile" || ret=$?
return $ret
}
start_service() {
reload_service
}
reload_service() {
AFFINITY=$AFFINITY_MIN
set_affinities set_affinity_per_cpu
}
stop_service() {
set_affinities set_affinity_shared
}
Then, enable it for future boots:
service irqbalance-manual enable
Also enable it immediately with:
service irqbalance-manual start
And in reply to @odhiambo, as @hnyman pointed out, this is the expected output: the IRQ balancing happens only once at boot and no service is kept running.
7 Likes
Suppose I flash an image and my router is soft-bricked. I start the router with an initramfs via a USB stick. Do I need to flash a sysupgrade or factory image ?
Use the factory image to ensure a full, clean flash.
I recently bricked mine (too much tinkering) but I knew I could easily recover with the USB stick which loads the initramfs thanks to the altered bootcmd.
To load the initramfs:
- Power down
- Insert USB stick with the fat/fat32 partition with the initramfs on it
- Power Up and wait about a minute
When the initramfs was loaded I used (win)scp to copy the factory image to the router and simply redid step 4 from the original instructions, flashing the factory image to both partitions :
4. Do a ubiformat to both rootfs partitions:
# ubiformat /dev/mtd18 -y -f /path_to/factory_image
# ubiformat /dev/mtd20 -y -f /path_to/factory_image
and I was back in business
Note:
- After the initramfs is loaded the router has an IP address of 192.168.1.1
- There is no LuCi, SSH into the router
- The default OpenWRT log in name is root and there is no password
- In this step you really need the factory image for ubi format, so the file with the .ubi extension from:
https://downloads.openwrt.org/snapshots/targets/ipq807x/generic/ - I copy the factory image to /tmp so the commands are then:
ubiformat /dev/mtd18 -y -f /tmp/openwrt-ipq807x-generic-dynalink_dl-wrx36-squashfs-factory.ubi
ubiformat /dev/mtd20 -y -f /tmp/openwrt-ipq807x-generic-dynalink_dl-wrx36-squashfs-factory.ubi
4 Likes
If you've just enabled the service...
then use service irqbalance-manual start to apply the settings without rebooting the router to do so.
1 Like
One very peculiar behavior that I noticed: I have now a DL-WRX36 and a Netgear WAX206 operating as APs. They're both using non-overlapping 160Mhz bands.
Testing with a Google Pixel 6 Pro, when I am connected to the DL-WRX36, my upload is capped in the 450-500 Mbps range, but download is in the 1.5Gbps area.
When I am connected to the WAX206, it's actually the opposite thing that happens - Upload is fine, reaching 1.5Gbps, but download now is the one getting capped in the 450-500 Mbps range.
This has been driving me crazy for a while, as I cannot find a good explanation on why this ~500Mbps cap happens in different sides depending on the AP.
That's pretty bizarre. Almost sounds like you are having non-wifi problems for it to present so symmetrically. May something didn't negotiate right, worn out connectors, chair sitting on a cable ( seen it happen ), etc. Try:
- reboot the switch
- check for interface errors at switch if its a manged switch and various nics in the path
- Try running iperf3 on the dynalink. I can get about 1.5 to 1.8 Gbps upload/ 2.35 download over my 2.5Gbps port to a machine with a 10G nic.
- Using any 2.5Gbps USB dongles in your testing path? On my laptop I've found them to have bandwidth limits in one direction. No idea why.
2 Likes