# Packet Steering question **URL:** \ **Category:** Hardware Questions and Recommendations\ **Created:** [August 23, 2020, 7:46am UTC](https://forum.openwrt.org/t/packet-steering-question/72458 "2020-08-23T07:46:57Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [August 23, 2020, 7:46am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/1 "2020-08-23T07:46:57Z")
I've trying investigate or find info what exactly doing `Packet Steering` ,i've installed htop, when i `chekd'ON Packet Steering`, gigabit connection uses 2 cores 20 and 30%, ~50% summary when i `chekd'OFF Packet Steering`, gigabit connection uses 2 cores 25 and 12%, ~40% summary Is this function share load or each core have full load? ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [December 28, 2022, 7:05pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/2 "2022-12-28T19:05:36Z")
Receive/Transmit Packet Steering is software based. It's functions (if the nic has more queues than the system's cpus e.g. ethtool -x eth0) is to help spread the network load across all processors. YMMV but my experience is that manually pinning an interrupt to a core & rps is very beneficial. [IRQ Affinity](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/performance_tuning_guide/s-cpu-irq) [Receive Packet Steering](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/performance_tuning_guide/network-rps) # bash script to set all receive/transmit queues to all processors ```plaintext find /sys/class/net/*/queues/[rt]x-[01234567]/[rx]ps_cpus -exec sh -c '[-w {}] && echo f > {} 2>/dev/null' \; ``` ---
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [December 29, 2022, 6:10am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/3 "2022-12-29T06:10:26Z")
2 years later) but thank you, i will try your script just today sir! ---
**Author:** ![open.nya](https://forum.openwrt.org/user_avatar/forum.openwrt.org/open.nya/32/32117_2.png) [@open.nya](https://forum.openwrt.org/u/open.nya)\ **Post date:** [December 29, 2022, 11:35am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/4 "2022-12-29T11:35:36Z")
> [@Sentenzo](#): > > doing `Packet Steering` Receive/Transmit Packet Steering is a _software_ technology intended for systems having NO _hardware_ packet processing acceleration. But it looks like your hardware can do it better. Server motherboard ASUS P9D-C/4L [https://www.asus.com/Commercial-Servers-Workstations/P9DC4L/specifications/](https://www.asus.com/Commercial-Servers-Workstations/P9DC4L/specifications/) Enterprise-level Intel I210-AT NIC's: [https://www.intel.com/content/www/us/en/products/sku/64400/intel-ethernet-controller-i210at/specifications.html](https://www.intel.com/content/www/us/en/products/sku/64400/intel-ethernet-controller-i210at/specifications.html) According to specification above, all of your NIC's supports _hardware_-accelerated network processing called RSS (Receive-Side Scaling). > **[8.6. Receive-Side Scaling (RSS) Red Hat Enterprise Linux 6 | Red Hat...](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/performance_tuning_guide/network-rss)** > > Access Red Hat’s knowledge, guidance, and support through your subscription. > Receive-Side Scaling (RSS), also known as multi-queue receive, distributes network receive processing across several hardware-based receive queues, allowing inbound network traffic to be processed by multiple CPUs. RSS can be used to relieve bottlenecks in receive interrupt processing caused by overloading a single CPU, and to reduce network latency. Intel I210-AT NIC's supports up to 4 hardware-based queues. To utilize RSS you can try to use Ethernet drivers from the following package repository: > **[GitHub - csrutil/realtek-r8125-openwrt: realtek 8125b 2.5G ethernet driver...](https://github.com/csrutil/realtek-r8125-openwrt)** > > realtek 8125b 2.5G ethernet driver for OpenWrt firmware - GitHub - csrutil/realtek-r8125-openwrt: realtek 8125b 2.5G ethernet driver for OpenWrt firmware ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 9, 2023, 9:34pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/6 "2023-01-09T21:34:45Z")
After some hardcore studying and research, my system with a j1900 is set to this simplified config: RSS is better than irq affinity or RPS, when the driver supports it. Disable irqbalance or packet steering. ```plaintext # rss defaults are not good. interfaces='eth0 eth1 eth2 eth3' for interface in $interfaces do ethtool -X "$interface" equal 2; ethtool -N "$interface" rx-flow-hash udp4 sdfn # udp4 uses only the addresses by default done ``` The journey and knowledge gained was well worth the effort. [Linux Scaling](https://www.kernel.org/doc/Documentation/networking/scaling.txt) [Tuning a very similiar unit](https://metebalci.com/blog/how-a-nic-rx-works/) ---
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [January 10, 2023, 5:02am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/7 "2023-01-10T05:02:47Z")
Thank you for sharing your experience! ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 6:36am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/8 "2023-01-10T06:36:07Z")
It sucks not being able to set the rxhash key, as I found what appears to be a good one with good flows and spreading per the author... ---
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [January 10, 2023, 7:02am UTC](https://forum.openwrt.org/t/packet-steering-question/72458/9 "2023-01-10T07:02:13Z")
can i ask what device you use with j1900? ---
**Author:** ![open.nya](https://forum.openwrt.org/user_avatar/forum.openwrt.org/open.nya/32/32117_2.png) [@open.nya](https://forum.openwrt.org/u/open.nya)\ **Post date:** [January 10, 2023, 1:04pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/10 "2023-01-10T13:04:54Z")
> [@mindwolf](#): > > when the driver supports it I don't have a x86 system myself, but I'm wondering if your network driver supports RSS? Did you have to install a special driver version with RSS support included? [https://github.com/csrutil/OpenWrt-NIC-Drivers](https://github.com/csrutil/OpenWrt-NIC-Drivers) Or maybe OpenWRT supports RSS on Intel NIC "out of the box" now? ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 2:41pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/11 "2023-01-10T14:41:54Z")
QOTOM-Q190G4N mini pc ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 2:48pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/12 "2023-01-10T14:48:14Z")
I believe even when I was using v19, it supported RSS ```plaintext DISTRIB_ID='OpenWrt' DISTRIB_RELEASE='22.03.2' DISTRIB_REVISION='r19803-9a599fee93' DISTRIB_TARGET='x86/64' DISTRIB_ARCH='x86_64' DISTRIB_DESCRIPTION='OpenWrt 22.03.2 r19803-9a599fee93' DISTRIB_TAINTS='' ``` ---
**Author:** ![open.nya](https://forum.openwrt.org/user_avatar/forum.openwrt.org/open.nya/32/32117_2.png) [@open.nya](https://forum.openwrt.org/u/open.nya)\ **Post date:** [January 10, 2023, 3:43pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/13 "2023-01-10T15:43:23Z")
> [@mindwolf](#): > > I believe even when I was using v19, it supported RSS Would you please to run the commands below and share us the results. This will show how many queues your NIC has. `ethtool -S eth0 | grep queue_` `cat /proc/interrupts` `cat /etc/modules.d/35-igb-intel` Thank you in advance! ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 4:04pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/14 "2023-01-10T16:04:27Z")
I think maybe you're confusing me with the OP, but here are my specs cpu: J1900 nic: I211-AT RSS queues: 2 rx 2 tx driver ver: ```plaintext version: 5.10.146 firmware-version: 0. 6-1 ``` I've tried to install the latest driver from the posted link, but it's giving me errors: ```plaintext root@ROUTER:~# opkg install /tmp/kmod-igb-intel_5.10.161+5.11.4-1_x86_64\ \(1\).ipk Unknown package 'kmod-igb-intel'. Collected errors: * pkg_hash_check_unresolved: cannot find dependency kernel (= 5.10.161-1-9309db0b721b9fb3fe826181264a78c4) for kmod-igb-intel * pkg_hash_fetch_best_installation_candidate: Packages for kmod-igb-intel found, but incompatible with the architectures configured * opkg_install_cmd: Cannot install package kmod-igb-intel. ``` ```plaintext root@ROUTER:~# cat /etc/os-release NAME="OpenWrt" VERSION="22.03.2" ID="openwrt" ID_LIKE="lede openwrt" PRETTY_NAME="OpenWrt 22.03.2" VERSION_ID="22.03.2" HOME_URL="https://openwrt.org/" BUG_URL="https://bugs.openwrt.org/" SUPPORT_URL="https://forum.openwrt.org/" BUILD_ID="r19803-9a599fee93" OPENWRT_BOARD="x86/64" OPENWRT_ARCH="x86_64" OPENWRT_TAINTS="" OPENWRT_DEVICE_MANUFACTURER="OpenWrt" OPENWRT_DEVICE_MANUFACTURER_URL="https://openwrt.org/" OPENWRT_DEVICE_PRODUCT="Generic" OPENWRT_DEVICE_REVISION="v0" OPENWRT_RELEASE="OpenWrt 22.03.2 r19803-9a599fee93" ``` ```plaintext root@ROUTER:~# ubus call system board { "kernel": "5.10.146", "hostname": "ROUTER", "system": "Intel(R) Celeron(R) CPU J1900 @ 1.99GHz", "model": "Default string Default string", "board_name": "default-string-default-string", "rootfs_type": "ext4", "release": { "distribution": "OpenWrt", "version": "22.03.2", "revision": "r19803-9a599fee93", "target": "x86/64", "description": "OpenWrt 22.03.2 r19803-9a599fee93" } } ``` ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 4:51pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/15 "2023-01-10T16:51:32Z")
DOH! I download and tried to install the wrong kernel version, Success! ```plaintext root@ROUTER:~# ethtool -i eth0 | grep version version: 5.11.4 firmware-version: 0. 6-1 expansion-rom-version: ``` now to give it a spin... ---
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [January 10, 2023, 4:53pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/16 "2023-01-10T16:53:12Z")
i've also tryed to install custom RSS driver from link provided, but it was not work great for me, **to fix your error, first delete kmod-igb already installed in your system, 100% will work))**, with this custom driver i had some strange cpu behaviour even my x86 router was working great.. ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 5:00pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/17 "2023-01-10T17:00:04Z")
Now I'm seeing only 1 queues for rx/tx vs 2. I'm not understanding the directions besides using ethtool to change the number of queues? ```plaintext root@ROUTER:/etc/modules.d# ethtool -l eth0 Channel parameters for eth0: Pre-set maximums: RX: 0 TX: 0 Other: 1 Combined: 2 Current hardware settings: RX: 0 TX: 0 Other: 1 Combined: 1 ``` ---
**Author:** ![Sentenzo](https://forum.openwrt.org/letter_avatar/sentenzo/32/5_5575768a8748004e209b776fc1b2916d.png) [@Sentenzo](https://forum.openwrt.org/u/Sentenzo)\ **Post date:** [January 10, 2023, 5:04pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/18 "2023-01-10T17:04:39Z")
i'm not sure how works that driver and why you have 1 less queue, but as i mentioned, i also had strange behaviour.. but with i210 but we can create a git-issue and ask developer)) ---
**Author:** ![mindwolf](https://forum.openwrt.org/user_avatar/forum.openwrt.org/mindwolf/32/3538_2.png) [@mindwolf](https://forum.openwrt.org/u/mindwolf)\ **Post date:** [January 10, 2023, 5:17pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/19 "2023-01-10T17:17:51Z")
So, to set the RSS no. of queues in openwrt for the I211-AT loadable module at boot. I believe the I210-AT has 4? - cd /etc/modules.d/ - edit the file 35-igb-intel with igb RSS=2,2,2,2 # this device has 4 nics, so we have 4 digits with (2) queues per nic... - save & reboot OR reload the module with the arguments ---
**Author:** ![anon66024873](https://forum.openwrt.org/letter_avatar/anon66024873/32/5_5575768a8748004e209b776fc1b2916d.png) [@anon66024873](https://forum.openwrt.org/u/anon66024873)\ **Post date:** [January 10, 2023, 7:02pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/20 "2023-01-10T19:02:04Z")
This is interesting stuff. I stumbled on that RSS github driver collection back in 2022 while I was searching for some related topics on QOS. I thought hardware acceleration works great in theory...but in real world its a big bag of unknown. Went to openwrt forums, nobody has created a topic of this until now.... Pretty cool to see you guys test if RSS hardware acceleration has some benefits -- most notably, if it improves CAKE qos. Will watch this thread with great interest. ---
**Author:** ![open.nya](https://forum.openwrt.org/user_avatar/forum.openwrt.org/open.nya/32/32117_2.png) [@open.nya](https://forum.openwrt.org/u/open.nya)\ **Post date:** [January 10, 2023, 7:39pm UTC](https://forum.openwrt.org/t/packet-steering-question/72458/21 "2023-01-10T19:39:21Z")
> [@mindwolf](#): > > So, to set the RSS no. of queues in openwrt for the I211-AT loadable module at boot. I believe the **I210-AT has 4**? It is not recommended to set the number of RSS queues greater than the number of **physical** processor cores: [https://access.redhat.com/documentation/en-us/red\_hat\_enterprise\_linux/6/html/performance\_tuning\_guide/network-rss](https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/performance_tuning_guide/network-rss) > When configuring RSS, Red Hat recommends limiting the number of queues to **one per physical CPU** core. Hyper-threads are often represented as separate cores in analysis tools, but configuring queues for all cores including logical cores such as hyper-threads has not proven beneficial to network performance. [https://www.kernel.org/doc/Documentation/networking/scaling.txt](https://www.kernel.org/doc/Documentation/networking/scaling.txt) > note that on processors with hyperthreading (HT), each hyperthread is represented asa separate CPU. For interrupt handling, HT has shown no benefit in initial tests, so limit the number of queues to the number of CPU cores in the system. This means that only 2 of the 4 available queues should be used when configuring RSS for I210-AT on a 2-Core CPU. [Next page](https://forum.openwrt.org/t/packet-steering-question/72458.md?page=2)