Deferring flow offload

[mle]

Hi,

I'm building a configuration where I'm logging some packets from the beginning of new flows using ulogd2 NFLOG input and PCAP output. This works nicely without offload.

Would it be possible to to defer offloading somehow by a couple of seconds or for a few pakets, so I'd be able to log some more interesting stuff from the beginning of the flow and have good perf for long running flows with the help of offloading?

I'm doing this to get beginning of flows to NFLOG:
nft 'insert rule inet fw4 forward ct state new,established ct bytes 0-10000 counter log group 1'

--Mika

[brada4]

Install ruleset.uc from https://github.com/openwrt/firewall4/pull/22/files
Then you can prepend accept rules in front of offload via /etc/nftables.d/anything.nft

chain handle_offload {
 ct bytes le 10000 counter log group 1 accept
}

[mle]

Thanks! I'll give this a try.

--Mika

[mle]

This seems to work just like I wanted. Thank you very much!

[brada4]

Save filename in /etc/sysupgrade.conf to keep it over upgrades and review that pull request (the part that helps you) if user there.

[mle]

Hmm, need to dig deeper still. Someting goes south with UDP as QUIC Client Hello fails to decode.

[brada4]

ct packets le 1000

[mle]

The going to 1000 packets did not help, but doing both

chain handle_offload {
 ct bytes le 10000 counter log group 1 accept

}

and

nft 'insert rule inet fw4 forward ct state new,established ct bytes le 10000 counter log group 1'

did help. I'm pretty new to netfilter, fw4 & offload combination details. What would be the best way to add the nft insert rule properly to fw4 config?

[brada4]

rule2 can go at start of offload where established,related is already filtered.

[brada4]

Understood - skip before offload, but logging should occur before forward table like prerouting to include "new" state ie first packet.

[mle]

Doing what you proposed in chain handle_offload works perfectly for TCP as far as I can tell; I'm able to control how much gets to NFLOG. For UDP however, chain handle_offload logging appears to produce only the reverse direction packets of a bi-directional flow; no packets from the flow initiator end up in the log. This is the likely cause of the failing QUIC SNI decode symptom I noticed earlier.

If I do the chain handle_offload and
nft 'insert rule inet fw4 forward ct state new,established ct bytes le 10000 counter log group 1'

I get the data I want, but there are a lot of duplicate packets logged.

If I do the chain handle_offload and
nft 'insert rule inet fw4 prerouting ct state new,established ct bytes le 10000 counter log group 1'
I seem to get the data I need and only some duplicates, so pretty OK.

Have not had to time to fully get my head aroung how all of this works as a system.

[brada4]

Yes, 1st udp bears SSL hello, if that is not logged to IDS it cannot re-construct tls connection start.
for TCP it comes 2 packets later so missing syn+synack is no problem.