Netfilter "Flow offload" / HW NAT


#61

It was said on another thread, that this should not be possible. It is confusing now.


#62

Software flow offload does not bypass the q-disc and does in fact work together with SQM. It is hw flow offload with which SQM is not compatible.


#63

I did not mean that they are incompatible and was referring to the bandwidth increase mentioned three posts above. Unless I am confused, this post below states that it is not possible.


#64

From what I understand, both can function at the same time and do not hinder each other. He is probably seeing a higher bandwidth in the SQM + flow offload scenario than SQM only, because the CPU is stressed less and hence is able to push & shape higher bandwidths.


#65

Yes, I guess it's going to vary from device to device but I'm currently doing 115 (which is what I configured SQM for out of a 120 total) from the 97-100 I was seeing with SQM alone in kernels 4.4 - 4.9. It could very well be placebo because my ISP has been down all day today and is probably pushing speeds so people don't complain.


#66

Well, I am not seeing any noticeable CPU utilization drop with flow offload and SQM...


#67

Flow offload seems to break IPSec (strongswan).


#68

did you tried this ?
https://git.openwrt.org/?p=openwrt/staging/nbd.git;a=commit;h=df87ab765855f98673e861a6d60b1ef8ff8979b5
and this is also posted earlyer


#69

i should clarify that this is without hardware flow offloading. it stops my firestick from being able to stream video, and it could be that this is also the source of the large number of connections.


#70

It seems that we could bypass wireguard interface, for example

iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -i pppoe-wan -j FLOWOFFLOAD


#71

@lucize

Unfortunately didn't solve it for me. Disabling flow offload or inserting an ACCEPT rule in the FORWARD chain before it makes it work.

@nbd

Any comment on this? I'm running strongswan (IKEv2) on my router and use latest OpenWrt git including your 650-netfilter-add-xt_OFFLOAD-target.patch patch.


#72

hmm...actually i have the problem with the firestick (but not the leaking connections) even if offloading is turned off. bouncing the firewall fixes it. i haven't had this problem with anything other apps/devices yet.


#73

Yes, PPPoE can be a bottleneck, it's why the Qualcomm NSS cores include PPPoE acceleration.


#74

Also for me, is that how it should work?


#75

As far as I understand, this shouldn't happen. These connections should properly time-out like they do without flow offload. Because eventually, the conntrack table will fill completely and you won't be able to open any more new connections. I haven't actually tried that yet myself though. I turned off flowoffload when I noticed this bug.

hw flow offload on mt7621 also offloads PPPoE. I am seeing 97-98% idle CPU usage while fully loading my 500/500 mbit connection. I was simply trying to figure out how I can keep the PPPoE offloading enabled and do the shaping on a dummy ethernet interface. Haven't had time yet to try this myself, but @dlakelan has a very interesting post regarding this on his blog: http://models.street-artists.org/2017/12/11/inbound-qos-with-virtual-ethernet-and-policy-routing/

Amazingly cool stuff :smiley:


#76

Thanks for your information.

based on that i created a little firewall.user script to do that on every interface except wireguard, it will offload pppoe-wan, wan, wan6, vpn interfaces, everything, just copy to the firewall.user file. If hw=1 then it is hw offloaded if 0 then software...

#network flow offload enable on ALL interfaces except wireguards
hw=1
ifaces=$(uci show network |grep =interface |sed "s|=.*||")
for i in $ifaces;do
	ifname=$(uci get $i.ifname)
	if [ ! "$(uci get $i.proto|grep wireguard)" ];then
		[ $hw ]&&hwswitch="--hw" ||hwswitch=""
		iptables -I FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -i $ifname -j FLOWOFFLOAD $hwswitch
	fi
done

#77

I can confirm my active connection count is much higher with flowoffload on than when it is off. Although for me, I consider it a small tradeoff for the kind of speeds I can get with it enabled.
Aside from this, it hasn't become so big that it's unmanageable: from 109 average to 460 average. I'm seeing 150 total and 70 average on the graphs. I'm also using unbound with Cloudflare's DNS-over-TLS which is known to keep a lot of connections active when using it so that could potentially be the reason as well.
Related question: I know I can increase the default 16384 shown in "active connections" because it is nothing but conntrack counts, right? A simple sysctl change should increase this number, but what are the side effects? higher RAM usage?


#78

Yes, but is minimal.


#79

Aren't active connections also a security risk in a sense? The firewall accepts packets for established and related connections. If connections linger around indefinitely, that might mean that even weeks later the firewall will accept these packets.

Also, eventually the conntrack table will be full and setting up new connections will be impossible.


#80

Wireguard + Flow offload should work together after this patch hits the master branch: http://lists.infradead.org/pipermail/openwrt-devel/2018-May/012675.html :slight_smile: