Qualcomm Fast Path For LEDE

quarky · February 18, 2018, 2:34am

SFE is designed to bypass the netfilter stacks processing once it has been established (i.e. after netfilter has determined that the connection is allowed.) This is similar in design to the first rule you see in the INPUT netfilter rules:

-A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT

The 'acceleration' is achieve by not going through the same netfilter checks for every incoming/outgoing network packets, as it is redundant, if it has been established previously to be safe.

So it doesn't really matter whether the traffic is going thru your router WAN link or through a tunnel to another location, SFE is supposed to 'accelerate' the connection.

I patched the fast-classifier code as the original codes does not work properly when it is used in conjunction with a not-physical network device and non-default routing table (in my case, an OpenVPN tunnel with Policy-Based-Routing.) The original code fails to find the correct output interface and when SFE kicks in, the network packets is discarded as it was sent to the wrong network interface.

I also encounter issues when using IPSec going through my router, whether it is directly via the router's WAN interface or via the OpenVPN tunnel. So I had to patch the IPv4 SFE code to bypass the IPSec traffic. AH traffic appears unaffected as far as I can tell. I guess it is because AH is done periodically, while IPSec traffic is more frequent and the SFE code does not track IPSec connection properly. Didn't spend too much time in this area tho. Planning to do it something in the future

My router is accelerating (it's running my custom DD-WRT build with my patched SFE codes, as it is a Broadcom router) traffic via OpenVPN tunnels just fine. A sample of the connection that goes to the WAN port and another that goes thru the tunnel is shown below from my router:

o=1, p=17 [40:cb:c0:xx:xx:xx]:192.168.28.134:37905 x.x.x.x:37905:[00:00:0c:xx:xx:xx] m=00000000 h=751
o=1, p=17 [00:00:00:00:00:00]:192.168.27.154:37905 x.x.x.x:37905:[00:00:00:00:00:00] m=00000000 h=8088

Both connections are IPSec traffic connections. First line is from local router (192.168.28.0/24) to WAN, while second line is from remote site (192.168.27.0/24) via an OpenVPN tunnel to WAN. What I noticed is that when traffic goes thru a tunnel, the MAC address is always 00:00:00:00:00:00, which makes sense as a virtual interface does not have MAC addresses.

For your case, it looks like the SFE code is still routing traffic through a physical interface, which is incorrect. So when SFE kicks in, the network packet gets discarded. I'm not too familiar with IPSec traffic routing in the Linux kernel so I'm not sure if I can help in your problem.

Maybe someone in the forum can help shed some light into how IPSec traffic are routed and we can collectively figure out how to solve this problem.

Mushoz · February 20, 2018, 5:42pm

An upstream version of offloading is ready in @nbd 's staging tree. More details can be found in his comment: https://github.com/lede-project/source/pull/1269#issuecomment-367056477

maurer · February 20, 2018, 7:05pm

this is the commit
https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=8d0c933b19dfa1f1fc38f685ca5925e0de7f83ce

Mushoz · February 20, 2018, 7:20pm

That is one of the commits that is needed. There are quite a few commits before that that are also needed for this to work AFAIK. Simplest would be to simply clone his staging tree and compile from there.

Sgt_Greco · February 26, 2018, 8:18am

I've installed the new image (Feb-2018) but SQM doesn't seem to work (again).
Speedtest and DSLreports results show full speed even though I throttle. Bufferbloat is awful.
Regular build works just fine.
Am I missing something?
(PPoE/VDSL)

rakesh · April 13, 2018, 7:20am

Is this confirmed that mwan3 will not work with fast-path acceleration?
Didn't find more info in this thread.

jgvega · April 13, 2018, 7:38am

rakesh · April 13, 2018, 8:29am

I am using mwan3 currently without fast-path enabled and it's working fine.
Question is: Does mwan3 work alongwith fast-path?
The comment I replied to claimed "mwan3 does not work with sfe".
On this I wanted more details if someone else tried it. Does it work for you?

jgvega · April 13, 2018, 10:52am

I went back to the version without fasth-path. I have overclocked the router

lucize · April 13, 2018, 2:07pm

I'm using mwan3 on 1043nd with this fastpath and is working

From 51fb6dbeecd0049b51776e7f3c13c31b80230244 Mon Sep 17 00:00:00 2001
From: Tan Hong Hui <hhtan72@yahoo-com>
Date: Sat, 27 Jan 2018 10:48:26 +0800
Subject: [PATCH] shortcut_fe: add sfe kernel module

Signed-off-by: Tan Hong Hui <hhtan72@yahoo-com>

bladeoner · April 15, 2018, 2:39pm

@gwlim when using make menuconfig I see some packages which you use and are included. Could you please tell me which packages are included which are not necessary to get Fast Path working?

I'm talking about packages like: DDNS, 3G support, UPnP thanks in advance.

philjohn · April 16, 2018, 11:36am

Doubtful - MWAN needs conntrack, SFE bypasses conntrack.

lucize · April 16, 2018, 8:42pm

the commit I posted help me achieve ~500mb/s while the 1043nd normally does only ~270mb/s

the PR @dissent1 posted is not working anymore after some time
Regards

bladeoner · April 21, 2018, 2:27pm

Did you create a new build recently?

I'm trying to create a build a firmware for the WNDR4300 but it keeps failing with both GCC 5 and GCC 6 on 17.01. What I did is following the "Patchset for MIPS74Kc AR71xx For LEDE Project" instructions and I only choose the packages I want to use plus kmod-fast-classifier, kmod-shortcut-fe and kmod-shortcut-fe-cm like @build000 did.

I think some thing changed in the latest kernel.

GCC 5 is giving me the errors described in the following Open Issue:
https://github.com/gwlim/mips24k-ar71xx-lede-patch/issues/12

Also the following PR didn't help:
https://github.com/gwlim/mips74k-ar71xx-lede-patch/pull/25

With GCC 6 i'm getting the errors described in the following Open Issue:
https://github.com/gwlim/mips24k-ar71xx-lede-patch/issues/9

If someone has any idea thanks in advance.

dchard · April 22, 2018, 10:02am

I would like to include SFE in my own build for ar71xx platform with kernel 4.9 (latest master). Can someone tell me which SFE version is the recommended version to use and where can I find it?

THanks!

sycohexor · April 22, 2018, 10:02am

whats the patch look like that you apply? i compile with 7.3 and 2.3 binutils works fine.

juppin · April 22, 2018, 5:32pm

Use this patch file for trunk...

github.com

juppin/openwrt_custom_builds_trunk/blob/master/patch/666-shortcut-fe_trunk.patch

From e2cb548b48b001bba25da1dfddf1aecc3fa7a973 Mon Sep 17 00:00:00 2001
From: Johann Neuhauser <johann@it-neuhauser.de>
Date: Tue, 15 May 2018 10:39:07 +0200
Subject: [PATCH] package: add Shortcut Forwarding Engine driver

Add an opensource Qualcomm Shortcut FE driver.

The driver consists of:
- sfe (core modules): shortcut-fe (ipv4) and shortcut-fe-ipv6 -
modules that actually do the offloading
- shortcut-fe-cm - simple shortcut-fe connection manager - a
module that creates offloading rules and passes to sfe
- fast-classifier - advanced connection manager - a module that
creates offloading rules and passes to sfe. Comparing to sfe-cm
has additional checks, connection statistics and creates an
offloading rule only when a connection hits 128 packets flowed
(can be adjusted through /sys/fast_classifier/offload_at_pkts)
- userspace-fast-classifier - a fast-classifier user-space tool
to manually add connections one want to offload
Both sfe-cm and fast-classifier get information about connections

This file has been truncated. show original

This is the one from dissent's pr on lede project

JochiPochi · April 26, 2018, 2:30pm

Hello there,

New dev here. was comparing speeds on the pre-made images @gwlim made for Archer V2 and V4

I'm simply using iperf2 in tcp mode over 30 sec intervals to measure the bandwidth from LAN to WAN and I get a difference in speed:
V2 -> 920 mbps
V4 -> 880 mbps

I would have expected the V4 to be faster.

So I grabbed the patch and compiled my own images to see if I could replicate the work.
@gwlim showed 1 issue with the pathes but @juppin's https://github.com/gwlim/mips74k-ar71xx-lede-patch/pull/25 worked great.
I used Lede 17.04 as by base and patched it include the Archer V4 files from upstream (Since the 17.XX branch doesn't have a V4)
And I get similar results:
V2 -> 920 mbps
V4 -> 860 mbps

@gwlim any thoughts on why this disparity might be occurring? I'm basically trying to get as close to gigabit as I can on the V4s.

If anyone has any ideas for optimization flags I can look at or any idea what so ever, I'm all ears

Thanks

slh · April 26, 2018, 4:38pm

There is nothing you can do, TP-Link simply cheapened the hardware for v4 (no dedicated CPU port for WAN, using the switch to segregate traffic over a single CPU port). v2 hardware is faster than v4, but for 1 GBit/s WAN throughputs you need something more capable than ar71xx anyways (x86, mvebu, maybe mt7621)

neheb · April 26, 2018, 6:08pm

Since you're close to gigabit, you could try overclocking to reach it. I believe it works well up to 1ghz.