SQM on Dynalink DL-WRX36

thekillah · March 21, 2023, 11:00am

I have a WRX36 running with the last snapshot (r22322).

I just can't get sqm to work. No matter what I set, only the upload and download speeds are affected.

Packet steering and software flow offloading are off.

I have a 400/12 MBit line.

And here is my configuration

config queue 'wan'
         option enabled '1'
         option interface 'wan'
         option download '380000'
         option upload '11400'
         option qdisc 'cake'
         option script 'piece_of_cake.qos'
         option linklayer 'ethernet'
         option debug_logging '0'
         option verbosity '8'
         option overhead '18

frollic · March 21, 2023, 11:04am

which other speeds should be affected ?

thekillah · March 21, 2023, 12:04pm

Only the down and upload will change.
It does not seem to have any influence on the other parameters.

moeller0 · March 21, 2023, 12:35pm

Well, what do you measure and what effect do you expect?

thekillah · March 21, 2023, 2:39pm

Sorry for my bad English.

I primarily measure the latency under load (full downstream and upstream).

Here are some example measurements (At this time of day I don't get full speed on my cable connection)

SQM Off

speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential -t 90
2023-03-21 15:28:09 Starting speedtest for 90 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..........................................................................................
 Download: 391.48 Mbps
  Latency: [in msec, 91 pings, 0.00% packet loss]
      Min:  15.823
    10pct:  70.172
   Median: 254.550
      Avg: 365.971
    90pct: 898.179
      Max: 1103.962
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 86 samples]
     cpu0:  40.1 +/-  6.3  @ 1465 MHz
     cpu1:   9.8 +/-  2.4  @ 1549 MHz
     cpu2:  10.1 +/-  3.0  @ 1444 MHz
     cpu3:  10.8 +/-  4.8  @ 1648 MHz
 Overhead: [in % used of total CPU available]
  netperf:  10.9
............................................................................................
   Upload:  12.17 Mbps
  Latency: [in msec, 92 pings, 0.00% packet loss]
      Min:  12.944
    10pct:  37.756
   Median:  46.441
      Avg:  46.590
    90pct:  51.241
      Max: 181.222
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 88 samples]
     cpu0:   1.5 +/-  1.3  @ 1403 MHz
     cpu1:   0.6 +/-  0.8  @ 1645 MHz
     cpu2:   0.7 +/-  1.0  @ 1620 MHz
     cpu3:   0.6 +/-  1.0  @ 1645 MHz
 Overhead: [in % used of total CPU available]
  netperf:   0.5

SQM on

speedtest-netperf.sh -H netperf-eu.bufferbloat.net -p 1.1.1.1 --sequential -t 90
2023-03-21 15:33:05 Starting speedtest for 90 seconds per transfer session.
Measure speed to netperf-eu.bufferbloat.net (IPv4) while pinging 1.1.1.1.
Download and upload sessions are sequential, each with 5 simultaneous streams.
..........................................................................................
 Download: 331.81 Mbps
  Latency: [in msec, 91 pings, 0.00% packet loss]
      Min:  15.465
    10pct:  53.155
   Median: 426.313
      Avg: 573.684
    90pct: 1056.605
      Max: 2457.269
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 87 samples]
     cpu0:  88.7 +/-  0.0  @ 1490 MHz
     cpu1:  13.0 +/-  4.6  @ 1665 MHz
     cpu2:  13.3 +/-  2.8  @ 1512 MHz
     cpu3:  12.2 +/-  2.3  @ 1674 MHz
 Overhead: [in % used of total CPU available]
  netperf:  22.7
...........................................................................................
   Upload:  11.73 Mbps
  Latency: [in msec, 91 pings, 0.00% packet loss]
      Min:  11.195
    10pct:  11.751
   Median:  13.338
      Avg:  14.529
    90pct:  15.253
      Max: 106.754
 CPU Load: [in % busy (avg +/- std dev) @ avg frequency, 88 samples]
     cpu0:   1.6 +/-  1.2  @ 1424 MHz
     cpu1:   0.6 +/-  0.9  @ 1657 MHz
     cpu2:   0.7 +/-  1.2  @ 1578 MHz
     cpu3:   0.9 +/-  3.0  @ 1654 MHz
 Overhead: [in % used of total CPU available]
  netperf:   0.5

frollic · March 21, 2023, 5:39pm

If you went from D to A, what's the problem?

thekillah · March 21, 2023, 6:30pm

My problem was the latency.

I have switched to qosify.
Now I have my dream result

moeller0 · March 21, 2023, 8:30pm

Have a look at irqbalance, you are likely overloading CPU0 and should try to distribute some of the load to the other CPUs.

thekillah · March 21, 2023, 8:34pm

irqbalance currently does not work on the WRX36

moeller0 · March 21, 2023, 8:39pm

How that? Anyway, you can try to set CPU affinity manually then? You really need to get some load of CPU0 or lower the shaper rate noticeably. (Also running speedtest.sh on the router is unlikely to help)

thekillah · March 21, 2023, 8:43pm

Have I already
/etc/rc.local

 #assign 4 rx interrupts to each cores
echo 8 > /proc/irq/50/smp_affinity
echo 4 > /proc/irq/51/smp_affinity
echo 2 > /proc/irq/52/smp_affinity
echo 1 > /proc/irq/53/smp_affinity

#assign 3 tcl completions to 3 CPUs
echo 4 > /proc/irq/73/smp_affinity
echo 2 > /proc/irq/74/smp_affinity
echo 1 > /proc/irq/75/smp_affinity

More is not possible at the moment

moeller0 · March 21, 2023, 8:44pm

What does cat /proc/interrupts show?

thekillah · March 21, 2023, 8:46pm

cat /proc/interrupts
           CPU0       CPU1       CPU2       CPU3
  9:          0          0          0          0     GIC-0  39 Level     arch_mem_timer
 13:    8637538   11657553   10536885   11147232     GIC-0  20 Level     arch_timer
 16:          2          0          0          0     GIC-0 354 Edge      smp2p
 17:          0          0          0          0     GIC-0 216 Level     4a9000.thermal-sensor
 18:          0          0          0          0     GIC-0 239 Level     bam_dma
 21:          0          0          0          0     GIC-0 270 Level     bam_dma
 22:          5          0          0          0     GIC-0 340 Level     msm_serial0
 23:      42079          0          0          0     GIC-0 178 Level     bam_dma
 24:          0          0          0          0     GIC-0  35 Edge      wdt_bark
 25:          0          0          0          0     GIC-0 357 Edge      q6v5 wdog
 29:          5          0          0          0     GIC-0 348 Edge      ce0
 30:     216882          0          0          0     GIC-0 347 Edge      ce1
 31:     122716          0          0          0     GIC-0 346 Edge      ce2
 32:      16288          0          0          0     GIC-0 343 Edge      ce3
 34:          0          0          0          0     GIC-0 443 Edge      ce5
 36:      18287          0          0          0     GIC-0  72 Edge      ce7
 38:          0          0          0          0     GIC-0 334 Edge      ce9
 39:          0          0          0          0     GIC-0 333 Edge      ce10
 40:          0          0          0          0     GIC-0  69 Edge      ce11
 47:          0          0          0          0     GIC-0 323 Edge      reo2ost-exception
 48:          9          0          0          0     GIC-0 322 Edge      wbm2host-rx-release
 49:          4          0          0          0     GIC-0 321 Edge      reo2host-status
 50:          0          0          0        694     GIC-0 320 Edge      reo2host-destination-ring4
 51:          0          0        583          0     GIC-0 271 Edge      reo2host-destination-ring3
 52:          0       2296          0          0     GIC-0 268 Edge      reo2host-destination-ring2
 53:        501          0          0          0     GIC-0 267 Edge      reo2host-destination-ring1
 57:      79741          0          0          0     GIC-0 263 Edge      ppdu-end-interrupts-mac3
 58:          0          0          0          0     GIC-0 262 Edge      ppdu-end-interrupts-mac2
 59:      74089          0          0          0     GIC-0 261 Edge      ppdu-end-interrupts-mac1
 60:          1          0          0          0     GIC-0 260 Edge      rxdma2host-monitor-status-ring-mac3
 61:          0          0          0          0     GIC-0 256 Edge      rxdma2host-monitor-status-ring-mac2
 62:          1          0          0          0     GIC-0 255 Edge      rxdma2host-monitor-status-ring-mac1
 63:          1          0          0          0     GIC-0 235 Edge      host2rxdma-host-buf-ring-mac3
 64:          0          0          0          0     GIC-0 215 Edge      host2rxdma-host-buf-ring-mac2
 65:          1          0          0          0     GIC-0 212 Edge      host2rxdma-host-buf-ring-mac1
 66:          0          0          0          0     GIC-0 211 Edge      rxdma2host-destination-ring-mac3
 67:          0          0          0          0     GIC-0 210 Edge      rxdma2host-destination-ring-mac2
 68:          0          0          0          0     GIC-0 209 Edge      rxdma2host-destination-ring-mac1
 73:          0          0       1162          0     GIC-0 191 Edge      wbm2host-tx-completions-ring3
 74:          0        393          0          0     GIC-0 190 Edge      wbm2host-tx-completions-ring2
 75:       1592          0          0          0     GIC-0 189 Edge      wbm2host-tx-completions-ring1
 77:         18          0          0          0     GIC-0  47 Edge      cpr3
 78:   14563296          0          0          0     GIC-0 377 Level     edma_txcmpl
 79:          0          0          0          0     GIC-0 385 Level     edma_rxfill
 80:   14631417          0          0          0     GIC-0 393 Level     edma_rxdesc
 81:          0          0          0          0     GIC-0 376 Level     edma_misc
 82:          0          0          0          0  pmic_arb 51380237 Edge      pm-adc5
 83:          0          0          0          0     smp2p   0 Edge      q6v5 fatal
 84:          1          0          0          0     smp2p   1 Edge      q6v5 ready
 85:          0          0          0          0     smp2p   2 Edge      q6v5 handover
 86:          0          0          0          0     smp2p   3 Edge      q6v5 stop
 87:          0          0          0          0   msmgpio  34 Edge      keys
 88:          0          0          0          0   msmgpio  63 Edge      keys
 89:      40588          0          0          0     GIC-0 172 Level     xhci-hcd:usb1
 90:         64          0          0          0     GIC-0 353 Edge      glink-native
IPI0:     23065      17735      19574      16695       Rescheduling interrupts
IPI1:   4073874   10007334    9773344    9976220       Function call interrupts
IPI2:         0          0          0          0       CPU stop interrupts
IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
IPI4:         0          0          0          0       Timer broadcast interrupts
IPI5:      3985       4037       3996       4429       IRQ work interrupts
IPI6:         0          0          0          0       CPU wake-up interrupts
Err:          0

hnyman · March 21, 2023, 8:48pm

Irqbalance heuristics does not recognize the ath11k interrupts (roughly 30 interrupts in use) and the edma interrupts that wired ports use.
See discussion in

github.com/Irqbalance/irqbalance

Not working with IPQ807x, all of the interrupts classified as class0

opened 07:29AM - 19 Feb 23 UTC

closed 07:43PM - 07 Mar 23 UTC

McGiverGim

Hi! The IPQ807x is now supported on OpenWrt. I have a Xiaomi AX3600 but this hap…pens too with others routers as seen in the official OpenWrt forum. I'm using `irqbalance` there with the latest master of OpenWrt, but the interrupts are all in the cpu0 and all are classified as class0. This is the data you need I think: ``` Version: 1.9.2-2 (as defined by OpenWrt) ``` ```bash #cat /proc/interrupts CPU0 CPU1 CPU2 CPU3 9: 0 0 0 0 GIC-0 39 Level arch_mem_timer 13: 1474020 1488109 1147576 1127555 GIC-0 20 Level arch_timer 16: 2 0 0 0 GIC-0 354 Edge smp2p 17: 0 0 0 0 GIC-0 216 Level 4a9000.thermal-sensor 18: 0 0 0 0 GIC-0 239 Level bam_dma 21: 0 0 0 0 GIC-0 270 Level bam_dma 22: 5 0 0 0 GIC-0 340 Level msm_serial0 23: 64315 0 0 0 GIC-0 178 Level bam_dma 24: 0 0 0 0 GIC-0 35 Edge wdt_bark 25: 0 0 0 0 GIC-0 357 Edge q6v5 wdog 29: 5 0 0 0 GIC-0 348 Edge ce0 30: 17601658 0 0 0 GIC-0 347 Edge ce1 31: 8673711 0 0 0 GIC-0 346 Edge ce2 32: 213398 0 0 0 GIC-0 343 Edge ce3 34: 31 0 0 0 GIC-0 443 Edge ce5 36: 190410 0 0 0 GIC-0 72 Edge ce7 38: 0 0 0 0 GIC-0 334 Edge ce9 39: 1 0 0 0 GIC-0 333 Edge ce10 40: 0 0 0 0 GIC-0 69 Edge ce11 47: 0 0 0 0 GIC-0 323 Edge reo2ost-exception 48: 1646168 0 0 0 GIC-0 322 Edge wbm2host-rx-release 49: 260 0 0 0 GIC-0 321 Edge reo2host-status 50: 801400 0 0 0 GIC-0 320 Edge reo2host-destination-ring4 51: 1376010 0 0 0 GIC-0 271 Edge reo2host-destination-ring3 52: 1702495 0 0 0 GIC-0 268 Edge reo2host-destination-ring2 53: 232679 0 0 0 GIC-0 267 Edge reo2host-destination-ring1 57: 5752486 0 0 0 GIC-0 263 Edge ppdu-end-interrupts-mac3 58: 0 0 0 0 GIC-0 262 Edge ppdu-end-interrupts-mac2 59: 10211035 0 0 0 GIC-0 261 Edge ppdu-end-interrupts-mac1 60: 1 0 0 0 GIC-0 260 Edge rxdma2host-monitor-status-ring-mac3 61: 0 0 0 0 GIC-0 256 Edge rxdma2host-monitor-status-ring-mac2 62: 1 0 0 0 GIC-0 255 Edge rxdma2host-monitor-status-ring-mac1 63: 1 0 0 0 GIC-0 235 Edge host2rxdma-host-buf-ring-mac3 64: 0 0 0 0 GIC-0 215 Edge host2rxdma-host-buf-ring-mac2 65: 1 0 0 0 GIC-0 212 Edge host2rxdma-host-buf-ring-mac1 66: 0 0 0 0 GIC-0 211 Edge rxdma2host-destination-ring-mac3 67: 0 0 0 0 GIC-0 210 Edge rxdma2host-destination-ring-mac2 68: 0 0 0 0 GIC-0 209 Edge rxdma2host-destination-ring-mac1 73: 3373884 0 0 0 GIC-0 191 Edge wbm2host-tx-completions-ring3 74: 525666 0 0 0 GIC-0 190 Edge wbm2host-tx-completions-ring2 75: 897200 0 0 0 GIC-0 189 Edge wbm2host-tx-completions-ring1 77: 3 0 0 0 GIC-0 47 Edge cpr3 78: 836656 0 0 0 GIC-0 377 Level edma_txcmpl 79: 0 0 0 0 GIC-0 385 Level edma_rxfill 80: 42784 0 0 0 GIC-0 393 Level edma_rxdesc 81: 0 0 0 0 GIC-0 376 Level edma_misc 82: 0 0 0 0 MSI 0 Edge PCIe PME, aerdrv 83: 0 0 0 0 pmic_arb 51380237 Edge pm-adc5 84: 0 0 0 0 smp2p 0 Edge q6v5 fatal 85: 1 0 0 0 smp2p 1 Edge q6v5 ready 86: 0 0 0 0 smp2p 2 Edge q6v5 handover 87: 0 0 0 0 smp2p 3 Edge q6v5 stop 88: 0 0 0 0 msmgpio 34 Edge keys 89: 31 0 0 0 MSI 524288 Edge ath10k_pci 90: 62 0 0 0 GIC-0 353 Edge glink-native IPI0: 3904 7179 5469 6472 Rescheduling interrupts IPI1: 819483 13763972 13900799 10592802 Function call interrupts IPI2: 0 0 0 0 CPU stop interrupts IPI3: 0 0 0 0 CPU stop (for crash dump) interrupts IPI4: 0 0 0 0 Timer broadcast interrupts IPI5: 706 830 804 766 IRQ work interrupts IPI6: 0 0 0 0 CPU wake-up interrupts Err: 0 ``` ```bash #irqbalance --debug This machine seems not NUMA capable. Prevent irq assignment to these isolated CPUs: 00000000 Prevent irq assignment to these adaptive-ticks CPUs: 00000000 Banned CPUs: 00000000 Package 0: numa_node -1 cpu mask is 0000000f (load 0) Cache domain 0: numa_node is -1 cpu mask is 0000000f (load 0) CPU number 3 numa_node is -1 (load 0) CPU number 1 numa_node is -1 (load 0) CPU number 2 numa_node is -1 (load 0) CPU number 0 numa_node is -1 (load 0) IRQ arch_mem_timer(9) guessed as class 0 IRQ arch_timer(13) guessed as class 0 IRQ smp2p(16) guessed as class 0 IRQ 4a9000.thermal-sensor(17) guessed as class 0 IRQ bam_dma(18) guessed as class 0 IRQ bam_dma(21) guessed as class 0 IRQ msm_serial0(22) guessed as class 0 IRQ bam_dma(23) guessed as class 0 IRQ wdt_bark(24) guessed as class 0 IRQ q6v5 wdog(25) guessed as class 0 IRQ ce0(29) guessed as class 0 IRQ ce1(30) guessed as class 0 IRQ ce2(31) guessed as class 0 IRQ ce3(32) guessed as class 0 IRQ ce5(34) guessed as class 0 IRQ ce7(36) guessed as class 0 IRQ ce9(38) guessed as class 0 IRQ ce10(39) guessed as class 0 IRQ ce11(40) guessed as class 0 IRQ reo2ost-exception(47) guessed as class 0 IRQ wbm2host-rx-release(48) guessed as class 0 IRQ reo2host-status(49) guessed as class 0 IRQ reo2host-destination-ring4(50) guessed as class 0 IRQ reo2host-destination-ring3(51) guessed as class 0 IRQ reo2host-destination-ring2(52) guessed as class 0 IRQ reo2host-destination-ring1(53) guessed as class 0 IRQ ppdu-end-interrupts-mac3(57) guessed as class 0 IRQ ppdu-end-interrupts-mac2(58) guessed as class 0 IRQ ppdu-end-interrupts-mac1(59) guessed as class 0 IRQ rxdma2host-monitor-status-ring-mac3(60) guessed as class 0 IRQ rxdma2host-monitor-status-ring-mac2(61) guessed as class 0 IRQ rxdma2host-monitor-status-ring-mac1(62) guessed as class 0 IRQ host2rxdma-host-buf-ring-mac3(63) guessed as class 0 IRQ host2rxdma-host-buf-ring-mac2(64) guessed as class 0 IRQ host2rxdma-host-buf-ring-mac1(65) guessed as class 0 IRQ rxdma2host-destination-ring-mac3(66) guessed as class 0 IRQ rxdma2host-destination-ring-mac2(67) guessed as class 0 IRQ rxdma2host-destination-ring-mac1(68) guessed as class 0 IRQ wbm2host-tx-completions-ring3(73) guessed as class 0 IRQ wbm2host-tx-completions-ring2(74) guessed as class 0 IRQ wbm2host-tx-completions-ring1(75) guessed as class 0 IRQ cpr3(77) guessed as class 0 IRQ edma_txcmpl(78) guessed as class 0 IRQ edma_rxfill(79) guessed as class 0 IRQ edma_rxdesc(80) guessed as class 0 IRQ edma_misc(81) guessed as class 0 IRQ PCIe PME, aerdrv(82) guessed as class 0 IRQ pm-adc5(83) guessed as class 0 IRQ q6v5 fatal(84) guessed as class 0 IRQ q6v5 ready(85) guessed as class 0 IRQ q6v5 handover(86) guessed as class 0 IRQ q6v5 stop(87) guessed as class 0 IRQ keys(88) guessed as class 0 IRQ ath10k_pci(89) guessed as class 0 IRQ glink-native(90) guessed as class 0 Adding IRQ 89 to database Adding IRQ 82 to database Adding IRQ 9 to database Adding IRQ 13 to database Adding IRQ 16 to database Adding IRQ 17 to database Adding IRQ 18 to database Adding IRQ 21 to database Adding IRQ 22 to database Adding IRQ 23 to database Adding IRQ 24 to database Adding IRQ 25 to database Adding IRQ 29 to database Adding IRQ 30 to database Adding IRQ 31 to database Adding IRQ 32 to database Adding IRQ 34 to database Adding IRQ 36 to database Adding IRQ 38 to database Adding IRQ 39 to database Adding IRQ 40 to database Adding IRQ 47 to database Adding IRQ 48 to database Adding IRQ 49 to database Adding IRQ 50 to database Adding IRQ 51 to database Adding IRQ 52 to database Adding IRQ 53 to database Adding IRQ 57 to database Adding IRQ 58 to database Adding IRQ 59 to database Adding IRQ 60 to database Adding IRQ 61 to database Adding IRQ 62 to database Adding IRQ 63 to database Adding IRQ 64 to database Adding IRQ 65 to database Adding IRQ 66 to database Adding IRQ 67 to database Adding IRQ 68 to database Adding IRQ 73 to database Adding IRQ 74 to database Adding IRQ 75 to database Adding IRQ 77 to database Adding IRQ 78 to database Adding IRQ 79 to database Adding IRQ 80 to database Adding IRQ 81 to database Adding IRQ 83 to database Adding IRQ 84 to database Adding IRQ 85 to database Adding IRQ 86 to database Adding IRQ 87 to database Adding IRQ 88 to database Adding IRQ 90 to database NUMA NODE NUMBER: -1 LOCAL CPU MASK: ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff,ffffffff ``` ```bash # for i in $(seq 0 300); do grep . /proc/irq/$i/smp_affinity /dev/null 2>/dev/null; done /proc/irq/1/smp_affinity:f /proc/irq/2/smp_affinity:f /proc/irq/3/smp_affinity:f /proc/irq/4/smp_affinity:f /proc/irq/5/smp_affinity:f /proc/irq/6/smp_affinity:f /proc/irq/7/smp_affinity:f /proc/irq/8/smp_affinity:f /proc/irq/9/smp_affinity:f /proc/irq/10/smp_affinity:f /proc/irq/11/smp_affinity:f /proc/irq/12/smp_affinity:f /proc/irq/13/smp_affinity:f /proc/irq/14/smp_affinity:f /proc/irq/16/smp_affinity:f /proc/irq/17/smp_affinity:f /proc/irq/18/smp_affinity:f /proc/irq/21/smp_affinity:f /proc/irq/22/smp_affinity:f /proc/irq/23/smp_affinity:f /proc/irq/24/smp_affinity:f /proc/irq/25/smp_affinity:f /proc/irq/29/smp_affinity:f /proc/irq/30/smp_affinity:f /proc/irq/31/smp_affinity:f /proc/irq/32/smp_affinity:f /proc/irq/34/smp_affinity:f /proc/irq/36/smp_affinity:f /proc/irq/38/smp_affinity:f /proc/irq/39/smp_affinity:f /proc/irq/40/smp_affinity:f /proc/irq/47/smp_affinity:f /proc/irq/48/smp_affinity:f /proc/irq/49/smp_affinity:f /proc/irq/50/smp_affinity:f /proc/irq/51/smp_affinity:f /proc/irq/52/smp_affinity:f /proc/irq/53/smp_affinity:f /proc/irq/57/smp_affinity:f /proc/irq/58/smp_affinity:f /proc/irq/59/smp_affinity:f /proc/irq/60/smp_affinity:f /proc/irq/61/smp_affinity:f /proc/irq/62/smp_affinity:f /proc/irq/63/smp_affinity:f /proc/irq/64/smp_affinity:f /proc/irq/65/smp_affinity:f /proc/irq/66/smp_affinity:f /proc/irq/67/smp_affinity:f /proc/irq/68/smp_affinity:f /proc/irq/73/smp_affinity:f /proc/irq/74/smp_affinity:f /proc/irq/75/smp_affinity:f /proc/irq/77/smp_affinity:f /proc/irq/78/smp_affinity:f /proc/irq/79/smp_affinity:f /proc/irq/80/smp_affinity:f /proc/irq/81/smp_affinity:f /proc/irq/82/smp_affinity:f /proc/irq/83/smp_affinity:f /proc/irq/84/smp_affinity:f /proc/irq/85/smp_affinity:f /proc/irq/86/smp_affinity:f /proc/irq/87/smp_affinity:f /proc/irq/88/smp_affinity:f /proc/irq/89/smp_affinity:8 /proc/irq/90/smp_affinity:f ``` The `lstopo-no-graphics` command is not found. I don't know if I can install it with some package. Thanks in advance!

and discussion around

and

Spacebar · September 29, 2023, 6:07pm

Thanks for this. I'm testing the same on a WAX630+WAX620 and still running after 2 days.
I did try ce0, ce1 etc... but that failed in a few seconds. Maybe running irqbalance and ban esN could work. I'll try later, but this is my daily driver I have to find the right time.

This is my adjustments:

# WAX630
echo 1 > /proc/irq/44/smp_affinity		// wbm2host-tx-completions-ring1
echo 2 > /proc/irq/49/smp_affinity		// wbm2host-tx-completions-ring2
echo 4 > /proc/irq/52/smp_affinity		// wbm2host-tx-completions-ring3

echo 8 > /proc/irq/30/smp_affinity		// edma_txcmpl
echo 4 > /proc/irq/32/smp_affinity		// edma_rxdesc

echo 8 > /proc/irq/57/smp_affinity		// ppdu-end-interrupts-mac1
echo 2 > /proc/irq/59/smp_affinity		// ppdu-end-interrupts-mac3
echo 4 > /proc/irq/61/smp_affinity		// ppdu-end-interrupts-mac2

I'm also testing Receive Packet Steering (RPS), since there's no RSS support on the NICs.

This is my adjustments on LAN1 (I have not verified this, but browsing feels snappier now):

cat /sys/class/net/lan1/queues/rx-*/rps_cpus
echo 1 > /sys/class/net/lan1/queues/rx-0/rps_cpus
echo 2 > /sys/class/net/lan1/queues/rx-1/rps_cpus
echo 4 > /sys/class/net/lan1/queues/rx-2/rps_cpus
echo 8 > /sys/class/net/lan1/queues/rx-3/rps_cpus

cat /sys/class/net/lan1/queues/tx-*/xps_cpus
echo 1 > /sys/class/net/lan1/queues/tx-0/xps_cpus
echo 2 > /sys/class/net/lan1/queues/tx-1/xps_cpus
echo 4 > /sys/class/net/lan1/queues/tx-2/xps_cpus
echo 8 > /sys/class/net/lan1/queues/tx-3/xps_cpus