Hello,
I have an Orange Pi R2S device (Ky X1 8-core RISC-V AI CPU).
The ONT is connected directly to the WAN port and PPPoE is configured.
IPv6 is disabled.
I tested the following configurations:
- Packet steering OFF, flow offload OFF
→ ~400 / 400 Mbps down/up - Packet steering ENABLED, flow offload OFF
→ ~780 / 900 Mbps down/up - Packet steering ENABLED (all CPUs), flow offload OFF
→ ~910 / 450 Mbps down/up - Packet steering ENABLE (normal, not all CPUs) or DISABLE, hardware flow offload ON
→ ~930 / 930 Mbps down/up
Software flow offloading does not seem to have any noticeable effect.
Enabling hardware flow offloading solves the throughput issue, but then SQM cannot be used.
Without hw-offloading, I cannot reach full gigabit speeds.
My questions are:
- Is it possible to achieve ~930 / 930 Mbps purely in software (CPU path) without enabling offload on this device?
- Why does upload speed drop significantly when packet steering is set to “enable all CPUs”?
- What is the best packet steering / offload combination for this device and for achieving symmetric gigabit speeds?
Thank you in advance for your help.
cat /proc/interrupts(with packet steering enable all cpu);
root@OpenWrt:~# cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3 CPU4 CPU5 CPU6 CPU7
11: 40698 32605 29054 26321 25115 53410 24441 26150 RISC-V INTC 5 Edge riscv-timer
12: 0 0 0 0 0 0 0 0 SiFive PLIC 60 Edge pinctrl
13: 0 0 0 0 0 0 0 0 SiFive PLIC 58 Edge x1-gpio
14: 6 0 0 0 0 0 0 0 SiFive PLIC 72 Edge pdma
15: 0 0 0 0 0 0 0 0 SiFive PLIC 36 Edge d4010800.i2c
16: 0 0 0 0 0 0 0 0 SiFive PLIC 37 Edge d4011000.i2c
17: 0 0 0 0 0 0 0 0 SiFive PLIC 38 Edge d4012000.i2c
18: 2198 0 0 0 0 0 0 0 SiFive PLIC 19 Edge d401d800.i2c
19: 0 0 0 0 0 0 0 0 SiFive PLIC 64 Edge spm8821
30: 0 0 0 0 0 0 0 0 spm8821 10 Edge rtc@pmic
57: 0 0 0 0 0 0 0 0 spm8821 37 Edge ky_pwrkey_rise
58: 0 0 0 0 0 0 0 0 spm8821 38 Edge ky_pwrkey_fall
59: 0 0 0 0 0 0 0 0 spm8821 39 Edge ky_pwrkey_skey
60: 0 0 0 0 0 0 0 0 spm8821 40 Edge ky_pwrkey_lkey
63: 0 0 0 0 0 0 0 0 SiFive PLIC 146 Edge x1-pcie
66: 0 0 0 0 0 0 0 0 PCI-MSI 134217728 Edge PCIe PME, aerdrv
67: 0 0 0 0 0 0 0 0 SiFive PLIC 147 Edge x1-pcie
70: 0 0 0 0 0 0 0 0 PCI-MSI 268435456 Edge PCIe PME, aerdrv
71: 0 0 0 0 0 0 0 0 SiFive PLIC 87 Edge JPU
72: 0 0 0 0 0 0 0 0 SiFive PLIC 86 Edge v2d-irq
73: 0 0 0 0 0 0 0 0 SiFive PLIC 42 Edge UART1
74: 0 0 0 0 0 0 0 0 SiFive PLIC 44 Edge UART3
76: 1700587 0 0 0 0 0 0 0 SiFive PLIC 131 Edge eth0
77: 724339 0 0 0 0 0 0 0 SiFive PLIC 133 Edge eth1
78: 0 0 0 0 0 0 0 0 SiFive PLIC 149 Edge dwc3-usb-wakeup
79: 0 0 0 0 0 0 0 0 SiFive PLIC 125 Edge xhci-hcd:usb2
80: 0 0 0 0 0 0 0 0 SiFive PLIC 105 Edge mv_udc, ehci_hcd:usb4
81: 0 0 0 0 0 0 0 0 SiFive PLIC 106 Edge usb-wakeup
82: 0 0 0 0 0 0 0 0 SiFive PLIC 118 Edge ehci_hcd:usb1
83: 0 0 0 0 0 0 0 0 SiFive PLIC 148 Edge usb-wakeup
84: 0 0 0 0 0 0 0 0 SiFive PLIC 74 Edge c0500000.linlon-v5
85: 0 0 0 0 0 0 0 0 SiFive PLIC 81 Edge x1ccic
86: 0 0 0 0 0 0 0 0 SiFive PLIC 82 Edge x1ccic
87: 0 0 0 0 0 0 0 0 SiFive PLIC 83 Edge x1ccic
88: 0 0 0 0 0 0 0 0 SiFive PLIC 84 Edge mars-cpp
89: 0 0 0 0 0 0 0 0 SiFive PLIC 79 Edge feisp
90: 1 0 0 0 0 0 0 0 SiFive PLIC 61 Edge thermal_zone0, thermal_zone1
91: 9048 0 0 0 0 0 0 0 SiFive PLIC 101 Edge mmc2
92: 0 0 0 0 0 0 0 0 SiFive PLIC 113 Edge ky-crypto-engine-0
93: 12 0 0 0 0 0 0 0 SiFive PLIC 52 Edge d4013400.mailbox
94: 0 0 0 0 0 0 0 0 PCI-MSI 134742016 Edge eth2-0
95: 0 0 0 0 0 0 0 0 PCI-MSI 134742017 Edge eth2-1
96: 0 0 0 0 0 0 0 0 PCI-MSI 134742018 Edge eth2-2
97: 0 0 0 0 0 0 0 0 PCI-MSI 134742019 Edge eth2-3
98: 0 0 0 0 0 0 0 0 PCI-MSI 134742020 Edge eth2-4
99: 0 0 0 0 0 0 0 0 PCI-MSI 134742021 Edge eth2-5
100: 0 0 0 0 0 0 0 0 PCI-MSI 134742022 Edge eth2-6
101: 0 0 0 0 0 0 0 0 PCI-MSI 134742023 Edge eth2-7
102: 0 0 0 0 0 0 0 0 PCI-MSI 134742024 Edge eth2-8
103: 0 0 0 0 0 0 0 0 PCI-MSI 134742025 Edge eth2-9
104: 0 0 0 0 0 0 0 0 PCI-MSI 134742026 Edge eth2-10
105: 0 0 0 0 0 0 0 0 PCI-MSI 134742027 Edge eth2-11
106: 0 0 0 0 0 0 0 0 PCI-MSI 134742028 Edge eth2-12
107: 0 0 0 0 0 0 0 0 PCI-MSI 134742029 Edge eth2-13
108: 0 0 0 0 0 0 0 0 PCI-MSI 134742030 Edge eth2-14
109: 0 0 0 0 0 0 0 0 PCI-MSI 134742031 Edge eth2-15
110: 0 0 0 0 0 0 0 0 PCI-MSI 134742032 Edge eth2-16
111: 0 0 0 0 0 0 0 0 PCI-MSI 134742033 Edge eth2-17
112: 0 0 0 0 0 0 0 0 PCI-MSI 134742034 Edge eth2-18
113: 0 0 0 0 0 0 0 0 PCI-MSI 134742035 Edge eth2-19
114: 0 0 0 0 0 0 0 0 PCI-MSI 134742036 Edge eth2-20
115: 0 0 0 0 0 0 0 0 PCI-MSI 134742037 Edge eth2-21
116: 0 0 0 0 0 0 0 0 PCI-MSI 134742038 Edge eth2-22
117: 0 0 0 0 0 0 0 0 PCI-MSI 134742039 Edge eth2-23
118: 0 0 0 0 0 0 0 0 PCI-MSI 134742040 Edge eth2-24
119: 0 0 0 0 0 0 0 0 PCI-MSI 134742041 Edge eth2-25
120: 0 0 0 0 0 0 0 0 PCI-MSI 134742042 Edge eth2-26
121: 0 0 0 0 0 0 0 0 PCI-MSI 134742043 Edge eth2-27
122: 0 0 0 0 0 0 0 0 PCI-MSI 134742044 Edge eth2-28
123: 0 0 0 0 0 0 0 0 PCI-MSI 134742045 Edge eth2-29
124: 0 0 0 0 0 0 0 0 PCI-MSI 134742046 Edge eth2-30
125: 0 0 0 0 0 0 0 0 PCI-MSI 134742047 Edge eth2-31
126: 0 0 0 0 0 0 0 0 PCI-MSI 268959744 Edge eth3-0
127: 0 0 0 0 0 0 0 0 PCI-MSI 268959745 Edge eth3-1
128: 0 0 0 0 0 0 0 0 PCI-MSI 268959746 Edge eth3-2
129: 0 0 0 0 0 0 0 0 PCI-MSI 268959747 Edge eth3-3
130: 0 0 0 0 0 0 0 0 PCI-MSI 268959748 Edge eth3-4
131: 0 0 0 0 0 0 0 0 PCI-MSI 268959749 Edge eth3-5
132: 0 0 0 0 0 0 0 0 PCI-MSI 268959750 Edge eth3-6
133: 0 0 0 0 0 0 0 0 PCI-MSI 268959751 Edge eth3-7
134: 0 0 0 0 0 0 0 0 PCI-MSI 268959752 Edge eth3-8
135: 0 0 0 0 0 0 0 0 PCI-MSI 268959753 Edge eth3-9
136: 0 0 0 0 0 0 0 0 PCI-MSI 268959754 Edge eth3-10
137: 0 0 0 0 0 0 0 0 PCI-MSI 268959755 Edge eth3-11
138: 0 0 0 0 0 0 0 0 PCI-MSI 268959756 Edge eth3-12
139: 0 0 0 0 0 0 0 0 PCI-MSI 268959757 Edge eth3-13
140: 0 0 0 0 0 0 0 0 PCI-MSI 268959758 Edge eth3-14
141: 0 0 0 0 0 0 0 0 PCI-MSI 268959759 Edge eth3-15
142: 0 0 0 0 0 0 0 0 PCI-MSI 268959760 Edge eth3-16
143: 0 0 0 0 0 0 0 0 PCI-MSI 268959761 Edge eth3-17
144: 0 0 0 0 0 0 0 0 PCI-MSI 268959762 Edge eth3-18
145: 0 0 0 0 0 0 0 0 PCI-MSI 268959763 Edge eth3-19
146: 0 0 0 0 0 0 0 0 PCI-MSI 268959764 Edge eth3-20
147: 0 0 0 0 0 0 0 0 PCI-MSI 268959765 Edge eth3-21
148: 0 0 0 0 0 0 0 0 PCI-MSI 268959766 Edge eth3-22
149: 0 0 0 0 0 0 0 0 PCI-MSI 268959767 Edge eth3-23
150: 0 0 0 0 0 0 0 0 PCI-MSI 268959768 Edge eth3-24
151: 0 0 0 0 0 0 0 0 PCI-MSI 268959769 Edge eth3-25
152: 0 0 0 0 0 0 0 0 PCI-MSI 268959770 Edge eth3-26
153: 0 0 0 0 0 0 0 0 PCI-MSI 268959771 Edge eth3-27
154: 0 0 0 0 0 0 0 0 PCI-MSI 268959772 Edge eth3-28
155: 0 0 0 0 0 0 0 0 PCI-MSI 268959773 Edge eth3-29
156: 0 0 0 0 0 0 0 0 PCI-MSI 268959774 Edge eth3-30
157: 0 0 0 0 0 0 0 0 PCI-MSI 268959775 Edge eth3-31
IPI0: 2159 2487 2582 2441 2417 2182 2696 2408 Rescheduling interrupts
IPI1: 9558 31765 385773 97839 206227 230585 93692 102282 Function call interrupts
IPI2: 0 0 0 0 0 0 0 0 CPU stop interrupts
IPI3: 0 0 0 0 0 0 0 0 CPU stop (for crash dump) interrupts
IPI4: 0 0 0 0 0 0 0 0 IRQ work interrupts
IPI5: 0 0 0 0 0 0 0 0 Timer broadcast interrupts
Even with packet steering set to "all CPUs", network interrupts still appear to be processed mainly by CPU0.
root@OpenWrt:/proc/irq/76# cat /proc/irq/76/smp_affinity
ff
root@OpenWrt:/proc/irq/76# cat /proc/irq/77/smp_affinity
ff
root@OpenWrt:/proc/irq/76#