[Solved] Nftables on OpenWrt and NAT

summers · October 13, 2018, 4:52pm

Has anyone managed to get nftables and nat to work on openwrt?

On my beagle farm I have

table ip nat {
	chain prerouting {
		type nat hook prerouting priority 0; policy accept;
	}

	chain postrouting {
		type nat hook postrouting priority 100; policy accept;
		masquerade
	}
}

And that works fine.

Now on my adsl router I've used

table ip nat {
	chain prerouting {
		type nat hook prerouting priority 0; policy accept;
	}

	chain postrouting {
		type nat hook postrouting priority 100; policy accept;
		oifname "pppoa-wan" masquerade
	}
}

But as soon as I switch off the iptables and just go with nftables - then I fail to get NAT and ability to route packets to the internet across my adsl router. Has anyone got this to work?

Oh yes - it isn't the output interface that is messing this up - it doesn't work if I remove that bit as well ...

jeff · October 14, 2018, 3:44pm

I haven't (yet) cut over to nftables on my OpenWrt installs as there are still bits of iptables that I haven't removed from my builds (they seem to be "locked in" by the build process). I have noted that there are several nftables-related modules. Checking that you have the ones related to NAT installed might be valuable.

Have you confirmed that your rules are being "taken" by nftables? nft -c and nft list ruleset may be helpful.

In working through nftables for another device, I found the logic behind the various "priorities" non-intuitive. The "timing" of NAT appears to need to be different for source and destination NAT. I found a table of existing priorities, such as in https://unix.stackexchange.com/questions/419851/when-and-how-to-use-chain-priorities-in-nftables very helpful.

Edit: The nftables docs do use priorities of 0 and 100, so my example of priorities following might be irrelevant.

Those docs also state:

Incompatibilities

You cannot use iptables and nft to perform NAT at the same time. So make sure that the iptable_nat module is unloaded:

% rmmod iptable_nat

From a running config of mine (Debian), I have the prerouting chain at priority -75 which my notes indicate as After MANGLE and NAT_DST, before FILTER

table ip nat4 {

     include "./defines_if.nft"

     include "./dnat_targets.nft"

     chain nat_rules_prerouting_ipv4 {
         type nat hook prerouting priority -75

         iifname $if_ext tcp dport ssh accept
         iifname $if_ext tcp dport $some_port dnat $some_target_IP
         # [...]

         return

     }

     chain nat_rules_postrouting_ipv4 {
         type nat hook postrouting priority 125

         oifname $if_ext snat $if_external_addr_ipv4

         return

     }

} # table ip nat4

summers · October 14, 2018, 4:15pm

I had the same thoughts about iptables modules in kernel. So I've just tried removing be hand so:

lsmod | egrep "^ip"
ip_tunnel              11360  1 sit

And that seems to have solved it! Or rather iptables is now down and I'm typing here!

The way I'm doing in, copied from my beagle farm, is I have the rules in /etc/nftables.conf. I then load using

nft -f /etc/nftable.conf

So far what I've been doing is just disabling iptables, before running nftables. That way I can just reboot if it goes wrong. Anyway have been confirming it correct with

nft list ruleset

And its just a copy of my /etc/nftables.conf - so my code is right ...

Anyway now its basically working, I'll de-install iptables. I'll set up a simple script to run the command that loads the nftables at boot.

If I get all I want working (NAT, firewall, forwarding, and ip6 goes out without nat) then I may look at how much effort to add to luci. If so I'll do it as a seperate table, as integrating nftables into fw3 seems far to complex to my mind ...

Oh yes - the priorities I'm taking from my beagle farm, which works. Think those numbers came from the documentation.

My full rules at the moment - which is fairly open, are:

table ip nat {
	chain prerouting {
		type nat hook prerouting priority 0; policy accept;
	}

	chain postrouting {
		type nat hook postrouting priority 100; policy accept;
		masquerade
	}
}
table inet filter {
	chain input {
		type filter hook input priority 0; policy drop;
		ct state { established, related } accept
		ct state invalid drop
		iifname "lo" accept
		iifname "br-lan" accept
		iifname "wlan0" accept
		ip protocol icmp accept
		ip6 nexthdr ipv6-icmp accept
		tcp dport ssh accept
		iifname "pppoa-wan" drop
	}

	chain forward {
		type filter hook forward priority 0; policy accept;
	}

	chain output {
		type filter hook output priority 0; policy accept;
	}
}

summers · October 15, 2018, 10:54am

Ah....

Disabling iptables isn't so easy. i removed modules I didn't want, but eventually the dependencies led back to luci-firewall, and that couldn't be deinstalled without removing luci. So left it in place.

Alas this had a problem, luci kept being spawned, until there were enough processes that the router ran out of memory and the oom killer kept killing things ...

So have now had to disable luci for the moment. Any ideas here, how to get luci running, but without the iptables firewall ?

Edit: More details - the process that was spawned something like a few hundered times was

/bin/sh /etc/rc.common /etc/init.d/dsl_control luci
flock 1000

Alas thats all the command I can give, only managed a "ps" and not a "ps w" before I had the reboot the router ...

jow · October 15, 2018, 11:01am

luci is just an empty meta package, you can simply remove it, along with luci-app-firewall (which is useless without iptables).

summers · October 15, 2018, 11:04am

Yes at the moment this is where I'm looking. Think this means I remove the whole web interface?

Now for me, this means just doing commands on the line interface. And actually I'm happy with that, e.g. I know enough of the low level commands to set everything up by hand.

But for wider users, is this the conclusion. e.g. if you want to stop using iptables, you have to disable the web interface? For most people, that would be an immediate turn off ...

jeff · October 15, 2018, 1:31pm

I take that to mean that the luci package is just dependencies, no "code" in and of itself. You would be able to "disable" LuCI in the build system, then hand-select the modules that you want for your GUI.

summers · October 15, 2018, 1:58pm

Ta thanks. yes that makes sense.

Bit more digging at this end though, doesn't look like it was the firewall causing luci to crash. or rather I switched on the http again - and its been stable for the last hour.

What I suspect it was, is there are some pipes in /tmp/pipe I believe set up by the adsl firmware. Now when I was on stock OS, knew that writing to these pipes and you could get the full adsl information, e.g. the noise and bits per frequency bin. Last night I tried to see if I could do the same on openwrt, but think this caused the problem. it looks like luci also access these pipes to get the dsl information - and somehow we clashed ....

summers · October 16, 2018, 4:23pm

Well I've set up nftables to do everything I used to do on the router before switching to openwrt. nftables took me some time to get used to after iptables, but I now find the interface easier. With nftables you can dump/load the whole set of rules in easily readable format, iptables only set up with a string of iptable commands.

So currently:

Filter all input into the router both IPv4 and IPv6
Do nat to the wan, on ipv4 only - ipv6 goes out unaltered and is routable
Grab connections to port 12345 on the router and forward to ssh on my NAS. This is needed for remote connection, as isp filters ports below 1024.

What I need to do

Add filtering to forwarding - if you treated my router as a gateway, and connected to a local ip address, you could avoid the firewall
Same problem with IPv6
problems at startup

The problems with startup are subtle, you can't load a rule into nftables before the interface exists. Now in particular the wan (pppoa-wan) takes a bit of time to come up, this means that if you mention pppoa-wan in the nftables, then they don't load. I've got round this by not mentioning pppoa-wan, but having the default action being what I want to happen with pppoa-wan - which is usually drop the packet. So any ideas how I can delay startup til after the pppoa-wan is up? I may do the rules in two halves, load up the basic firewall early; then add the pppoa-wan commands later. But same question how do I delay a task until after the interface comes up?

My current configuration for people interested:

table ip nat {
	chain prerouting {
		type nat hook prerouting priority 0; policy accept;
		tcp dport 12345 dnat to 192.168.2.111:ssh
	}

	chain postrouting {
		type nat hook postrouting priority 100; policy accept;
		masquerade
	}
}
table inet filter {
	chain input {
		type filter hook input priority 0; policy drop;
		ct state { established, related } accept
		ct state invalid drop
		iif "lo" accept
		iif "br-lan" accept
		iif "wlan0" accept
		ip protocol icmp accept
		ip6 nexthdr ipv6-icmp accept
		tcp dport ssh accept
	}

	chain forward {
		type filter hook forward priority 0; policy accept;
	}

	chain output {
		type filter hook output priority 0; policy accept;
	}
}

jeff · October 16, 2018, 5:21pm

Try using [i|o]ifname instead of [i|o]if

That was the only "reasonable" way I could get my rules to load before the interfaces came up, especially with VLANs involved.

summers · October 16, 2018, 5:26pm

@jeff Good idea - yes that sounds like it would work, so it looks up the name when doing packets rather than when the rule is loaded. Must admit I automatically switched to [i|o]if as its lower load on the system ...

craigphicks · April 19, 2019, 8:24am

Your nftables configuration was helpful. However one part that didn't work in my case was your line

masquerade

In my case, with that setting, masquerade was also applied to packets destined for lo. That caused dns packets from dnsmasq to get lost on the last leg back to the caller, e.g., ping google.com.
Changing to

iifname "br-lan" masquerade

fixed that. (Currently "br-lan" is the only active external interface.)

The full listing

table ip nat {
	chain prerouting {
		type nat hook prerouting priority 0; policy accept;
	}

	chain postrouting {
		type nat hook postrouting priority 100; policy accept;
		oifname "br-lan" masquerade
	}
}
table ip filter {
	chain input {
		type filter hook input priority 0; policy drop;
		ct state { established, related } accept
		ct state invalid drop
		iif "lo" accept
		ip protocol icmp accept
		tcp dport ssh accept
	}
	chain forward {
		type filter hook forward priority 0; policy drop;
	}
	chain output {
		type filter hook output priority 0; policy drop;
		ct state { established, related, new } accept
		ct state invalid drop
	}
}

summers · April 19, 2019, 10:41am

The trick with the masquerade line is oiftype ppp masquerade so it should only operate on ppp interfaces. if you do ip a look for the link/ppp - those are the only interfaces it should operate on. Sounds strange that ppp is on lo, or even on br-lan; so suspect your set up is different to mine. Trick you need a way of identifying the interface on which you want to do NAT - I used the ppp connection, you probably need something else, and interface name is fine. IIRC going by name, uses a tad more resources, as it need to look up the name IIRC every time ...

Oh yes also worth saying, probably easier working from my documentation - that was the latest version: openwrt nftables documentation

tmomas · April 29, 2019, 10:41am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.