I think that's more suited to a server environment vs a router in this case. I've test 1 vs 2 with the default hash of equal 1 or weight 0 1, and the default of 2 pairs with equal 2(spreading the hash more even without a hash key) is more profitable. 1 pair seems cause more cpu usage on only 2 cores vs across them. I guess the jury is still out on that. My tests have been from nperf.com and waveform.
new driver error from that link
Cannot get RX network flow hashing options: Not supported
This means it's not spreading across the cores effectively.
I'm currently using stock v22.03.3 combined squash as ANY EFI build has weird errors ranging from stalled reboots to loading issues.
RSS=x,x,x,x has no effect on stock. still using my ethtool script.
RSS is enabled but only select distrubtions such as Mellanox or Nvidia are actually taking advantage of the hash that assigns the cores to cpus, thus RSS should be assigned 1 queue per cpu and utilize RPS and RFS.
By default linux supposedly uses the sysct /proc/sys/net/core/netdev_rss_key but this is not the case as the drivers don't take advantage of this either, so RSS is kinda useless if you can't guide the system on how to direct the queues.
Recompiled with these options, yet not all in my configs are listed. Only the main options to use for reference. This might help someone with a similiar platform to get the most out of there unit beyond the "safe" defaults. My unit is running very stable.
This version runs ultra fast compared to the later versions. I'm guessing more fluff and kernel bloat?
CONFIG_NO_HZ_IDLE=y
CONFIG_HZ_1000=y
CONFIG_HZ=1000
CONFIG_PREEMPT_VOLUNTARY=y
CONFIG_MATOM=y #This is actually the closest architecture to my J1900 with general instructions.
Simple script placed in /etc/hotplug.d/net/20-tuning. The sleep parameter is needed because of a race condition when booting. I haven't found a better solution to implement earlier but not before devices are loaded.
#!/bin/sh
# this will explain the layout of the processors for the j1900 quantum & similar units
# cpu(eth0) is know to the system as eth1 hence (1)
# cpu0= (1) | cpu1= (2) | cpu2= (4) | cpu3= (8)
# each line will pin the system irq's
awk '/eth0/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
awk '/eth1/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
awk '/eth2/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
awk '/eth3/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
awk '/i915/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
awk '/ahci/ { gsub(/:/,""); print $1}' /proc/interrupts | while read i; do echo "e" > /proc/irq/$i/smp_affinity;done
# (the default for openwrt v19 is powersaving, which works just fine). change the scaling governor to performance instead of powersaving
#find /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor -exec sh -c 'echo performance > {}' \;
sleep 15
# configure ethtool RSS to spread among the cores
for NICS in eth0 eth1 eth2 eth3; do
ethtool -X $NICS equal 2
done
# the default cpu is 0, which we want to exclude so as to utilize the other processors. this sets each receive and transmit packet steering element rps_cpus to (2+4+8), or e (the hexadecimal value for 14).
find /sys/class/net/*/queues/rx-[01]/rps_cpus -exec sh -c '[ -w {} ] && echo "e" > {} 2>/dev/null' \;
Are you still running with these options set? Do you have any benchmarks to show before & after performance with these set vs defaults?
Also, where did you set these values and how did you confirm they were built against your firmware image? I am struggling with testing this myself and hope you can help me with some answers
The options I specified are used in make kernel_menuconfig, which are chosen in the gui prompt. As I understand it, there are several places that the kernel config pulls from and I honestly don’t remember which directories they are located in to manually edit those files. The .config is mainly for the system config but not necessarily the Linux kernel itself.
