No routing/NAT, the measurements were between just the Box and my PC.
And I actually tried it both ways, Box as server, then switched to PC as server, the results are the same.
IMHO flow offloading is not targeted for the second usecase.
It's not?
It makes a huge difference with the same tests on my EdgeRouter X, although I understand that it is much beefier than xrx200.
I believe flow offloading really just helps speeding up processing of packets that are traversing the router, packet originating or terminating at the router should not see a big improvement, as far as I know.
Well, I've just read up on it again and that is what it should do, true.
But for some reason the tests above are about 100 Mbits/s faster on MT7621.
TCP is 776 Mbits/s with offloading and 674 Mbits/s without.
UDP is 323 Mbits/s with offloading and 234 Mbits/s without.
I ran the test just now.
No idea what is going on there, maybe some side effect of sorts?
Anyway, sorry for getting off topic.
MT7621 is 880MHz vs ~500MHz in xrx200 and MT7621 has 256kB L2 cache. The xrx200 doesn't have any L2 (IMO) and so its system bus is most likely saturated from that iperf testing.
Probably, yeah.
But does that explain why software flow offloading makes a difference on MT7621 when it shouldn't?
The wiki says:
Enable software flow offloading for connections. (decrease cpu load / increase routing throughput)
So, with no routing taking place it should not do anything, but does.
Hello again,
I've managed to get my patches into vanilla kernel about month back, so the openwrt will start use them automatically when it gets to next future stable branch (5.4+ probably, .. long time ahead 
).
But about a week ago one of my patch was backported to all stable trees including the one openwrt is using, so I guessed I could rebase my patches.
The autobalancing function was removed for the vanilla, so you must to change the interrupts by hand, for example:
echo 1 > /proc/irq/73/smp_affinity
echo 2 > /proc/irq/75/smp_affinity
(the value is bitmask, just google "kernel bitmask affinity" or something like this).
Copy the files into target/linux/lantiq/patches-4.19 of your openwrt directory and reconfigure the kernel. The kernel I based this is 4.19.65 and the openwrt is current git version (old about a day). Don't forget pastebin may eat the last empty line.
Too bad it was a great thing for the router. It automatically set the interrupts between the two VPEs but no more. Every thing is mapped to the VPE 0 even though the smp_affinity settings are set to 3 meaning use both cpus.
Well there were objections towards it. And the switching VPE every interrupt event has big overhead (the code must migrate from one VPE to the other). There is always possibility to fix irqbalance daemon, one time remapping of the interrupts between cores or just patch the balancing code (just few lines).
As of now, the wifi performance is identical whether the IRQ balancing is working or not. IMO it wouldn't really make a big difference in performance because CPU gets utilized in full if you use Samba v4 and at the moment I did an iperf3 test for 2 minutes and I am getting 56MB/s avg for receiving data from router and 110MB/s for sending data to the router. As I am using my phone I cant really test whether the CPU is being utilized in full but the load average was about 2.5 for a 2 minute udp data transfer each way separately.
So I've been using your secondary VPE IRQ patches successfully for a while - just manually punting vrx200_rx, ath9k and ath10k onto the second CPU.
Since your earlier patches have been forgotten in the mailing list and your revised (non-auto balancing) patches are now upstream (well done on that), have you considered raising a pull request on OpenWRT github for including these in kernel 4.19 on git?
Just been dealing with the packet steering script issue, which mostly affects VDSL2 users who should be getting above 100Mb/s. See PR2553 for fixing it to use all CPUs but also disable it by default (in preference to proper IRQ balancing).
The pull request is a nice idea so far. I believe it may get added soon. I hope they also add it to v19.07 before it gets released.
Well, 19.07.1 is performing fine for LAN->DSL WAN vrx200/HH5A with the packet steering scripts and software flow offloading, however due to the ptm and ath10k pinning to CPU0, the WLAN0->DSL WAN often tops out ~85 MB/s, only sometimes reaching 100-104 MB/s to tease that it is capable.
I believe manually punt Ath10k away from DSL (ptm_mailbox_isr) will fix this, so I am going to try and test these patches against 19.07.1; Has anyone used these with 18.07.1 for HH5A sucessfully yet to fix 5G WiFi->DSL performance?
root@HH5A_40F2012D8A78:~# cat /proc/interrupts
CPU0 CPU1
7: 723210 727868 MIPS 7 timer
8: 2092 53215 MIPS 0 IPI call
9: 11048 67476 MIPS 1 IPI resched
30: 0 0 icu 30 ath9k
62: 0 0 icu 62 1e101000.usb, dwc2_hsotg:usb1
63: 150912 0 icu 63 mei_cpe
72: 9 0 icu 72 vrx200_rx
73: 160 0 icu 73 vrx200_tx
75: 0 0 icu 75 vrx200_tx_2
96: 156948 0 icu 96 ptm_mailbox_isr
112: 258 0 icu 112 asc_tx
113: 0 0 icu 113 asc_rx
114: 0 0 icu 114 asc_err
126: 0 0 icu 126 gptu
127: 0 0 icu 127 gptu
128: 0 0 icu 128 gptu
129: 0 0 icu 129 gptu
130: 0 0 icu 130 gptu
131: 0 0 icu 131 gptu
144: 418556 0 icu 144 ath10k_pci
161: 2 0 icu 161 ifx_pcie_rc0I am going to test the patches now with v19.07.1 but I do not use the 5G WiFi so cannot comment on the performance of that one.
I have bad expirience in VDSL -> Wlan using 4.19.65 patches on W8970
0901-add-icu-smp-support.patch
0902-icu-devicetree.patch
0904-xrx200-net-smp-frags-support.patchw
It barely touch 23Mb in speedtest net even with software offload and irq mapping . Got ~30Mb in 19.0.7.1 software offloading and packet steering
@achmar16 I'm looking forward to test irq-smp patches with recent 19.07.01
You need to understand that 0904 patch is for Ethernet driver and not for VDSL. I think you need to check your LAN to LAN performance on the router itself. For that use two instances of iperf3 in TCP mode on 127.0.0.1, one would be the server and the other would be the client on the same router. You can also check your CPU load while iperf3 test is happening through htop. I am using the patches 0901 and 0902 with v19.07.1 and the best I am getting is 4mb/s with Samba v2 and 100% CPU load while transferring the files.
One more thing, if you want increased speed then I suggest you change your build environment to v18.06.x and apply the patches. AFAIR I was able to get 10mb/s at some point in v18.06.
Yes I know .... 0904 can be unrelated to low vdsl to wlan performance . There is no problem when client is connected via ethernet with vdsl2 @60Mbit I have
OK testing on current (less than week back) snapshot of openwrt, kernel 5.4 (testing/experimental). For dual VPE IRQ you need just this patch:
0901-add-icu-smp-support.patch
everything else regarding to the dual VPE interrupt driver is in vanilla by now.
@pc2005 I'm using these patches on v19.07.2 with kernel 4.14:
But 0901-add-icu-smp-support.patch failes:
Applying /home/toni/openwrt/target/linux/lantiq/patches-4.14/0901-add-icu-smp-support.patch using plaintext:
(Stripping trailing CRs from patch; use --binary to disable.)
patching file arch/mips/lantiq/irq.c
Hunk #6 FAILED at 227.
Hunk #7 succeeded at 273 (offset 1 line).
Hunk #8 succeeded at 311 (offset 1 line).
Hunk #9 succeeded at 334 (offset 1 line).
patch unexpectedly ends in middle of line
Hunk #10 succeeded at 380 with fuzz 1 (offset 1 line).
1 out of 10 hunks FAILED -- saving rejects to file arch/mips/lantiq/irq.c.rej
Patch failed! Please fix /home/toni/openwrt/target/linux/lantiq/patches-4.14/0901-add-icu-smp-support.patch!
Makefile:23: recipe for target '/home/toni/openwrt/build_dir/target-mips_24kc_musl/linux-lantiq_xrx200/linux-4.14.171/.prepared_eb9ceb9d5cf211fcd2a83260894be243' failed
Anything I'm doing wrong here? Or is this patch not suited for 4.14.171?
Try this. It may work for you with 4.14 I guess or maybe with v4.19 too.
They work. Thank you. I guess 0903-add-icu1-node-for-smp.patch then isn't need?