I have those enabled. try putting them back, they load x_tables and ip_tables and ip6_tables
summers
82
Sigh. I spoke too quickly. DNS keeps dying IPv6 6to4 doesn't work, unless I totally open the firewall, and only do NAT via nftables.
I've now installed every nft package, and removed all iptables in /etc/modules.d/; viz:
- kmod-nft-arp - 4.14.162-1
- kmod-nft-bridge - 4.14.162-1
- kmod-nft-core - 4.14.162-1
- kmod-nft-fib - 4.14.162-1
- kmod-nft-nat - 4.14.162-1
- kmod-nft-nat6 - 4.14.162-1
- kmod-nft-netdev - 4.14.162-1
- kmod-nft-offload - 4.14.162-1
- libnftnl11 - 1.1.4-1
- nftables - 0.9.2-1
and
- 42-ip6tables
- ipt-conntrack
- ipt-core
- ipt-nat
- ipt-offload
I think all I can conclude in on 19.07 and the TP-Link W8970 router that nftables is badly broken.
So all I can see as the way forward is to download the source, and compile it myself, and do a fresh install.
Sigh - what a mess.
1 Like
can you try a minimalist firewall? Something like my script above? Perhaps it's an incompatibility in your firewall script, such that nftables in general broke your firewall design, and it has nothing to do with OpenWrt's build per se?
summers
84
My /etc/nftables.conf is already fairly minimal - shown below:
table ip nat {
chain prerouting {
type nat hook prerouting priority 0; policy accept;
tcp dport 12345 dnat to 192.168.2.111:ssh
}
chain postrouting {
type nat hook postrouting priority 100; policy accept;
oiftype ppp masquerade
}
}
table inet filter {
chain input {
type filter hook input priority 0; policy drop;
ct state { related, established } accept
ct state invalid drop
iiftype != ppp accept
ip protocol icmp accept
ip6 nexthdr ipv6-icmp accept
iiftype ppp drop
}
chain forward {
type filter hook forward priority 0; policy drop;
ct state { related, established } accept
ct state invalid drop
iiftype != ppp accept
ip protocol icmp accept
ip6 nexthdr ipv6-icmp accept
tcp dport ssh accept
iiftype ppp drop
}
chain output {
type filter hook output priority 0; policy accept;
}
}
I could drop out some ipv6 stuff, and the remote ssh connection, but thats about all I could do.
Yeah, that's minimal enough that I don't think its likely they broke it in nftables itself.
Let us know if a custom compile helps!
summers
86
Accidental breakthrough this morning, last few days when the connection goes down, and I need the internet, I've been totally switching off the firewall (except for NAT) - and that gave me a couple of hours until people started scanning the router.
Anyway this morning I did nft flush ruleset ; nft -f /etc/nftables.conf to open up the firewall, but actually the config for that was in /tmp/nftables.conf - so actually I just reloaded the same nftables that were in the kernel as before, just with a "flush ruleset" in between. Now is what interesting is that was enough to get the firewall working again, even though nothing changed. 5 minutes ago I did the same, and got the same results.
This makes it sound like something in the kernel is filling up, this means I can't do the atomic replacement of the tables, and that it dies after a bit.
Anyway just busy downloading the source. I then compile a openwrt that doesn't have and iptables in, but is compiled with nftables in the main kernel. So only really a OS for me ...
1 Like
Any luck with this? Also what device are you testing on? The flush ruleset in the script works fine on my x86 virtual machine so I'm wondering if there are architecture specific bugs?
summers
88
Its a TP-Link TD-W8970 v1 device which is a lantiq xrx200 device, or MIPS 54Kc IIRC.
I started the build last night, had no time to check this morning, but my desk top is a 10 year old machine, and a slow one at that (AMD E350). Once it completes its build, I'll need to dig into how it has built the kernel, and see if any odd settings.
Doesn't help that openwrt doesn't have a /proc/config.gz that you can use to check the exact settings.
I'll probably then do some hacking (e.g. in standard build can't disable iptables) so I know exactly what I've got. Really don't want to flash it until I'm sure I have something that works.
Actually bit of digging and kmod-ikconfig may give me /proc/config.gz
pinging this to see if you had any further info getting nftables running.
summers
90
I git clone openwrt, and set it building openwrt. Takes about a day to compile on my old machine.
First build barffed, switched on more logging, it was clear that the git clone had moved on the linux kernel. Also couldn't complete the build, as my images were too large to be able to be set up for my machine.
So I've now rechecked out what I hope is the 19.07 git clone. And am busy building what I hope should be the image created for the official release. If that works, and it stays inside the size limit, I'll then look at the changes needed.
On the first attempt, I went through the kernel configuration, attempting to take out all ip tables, and switch on all nf tables. nf tables clearly needs some of the netfilter stuff which it depends on. Couldn't find anything that set a memory size in firewalls, bit surprised as IIRC nftables runs as a virtual language in the kernel (BPF - Berkley Packet Filter).
Anyway as said, I expect it to be a slow process; these things are quite involved, and it must be almost 10 years since I last looked at the openwrt build process - so there is some learning to do.
But I'm still on the case.
Oh yes kmod-ikconfig does give the kernel configuration, a quick browse and it all looked reasonable - but its hard from the config file as it doesn't show dependencies. Really need to load the config file into a kernel menuconfig to see all the cross dependencies ...
Need one of those preemptable google cloud instances with 64 cores. Costs like $3 to run for 4 hours. 
summers
92
OK compiled 19.07 - the sysupgrade just over 5MB (5112599), which is a tad smaller than the release.
So now went it, took out firewall support, and turned on almost everything net related in the kernel.
Now building again, whilst it builds I compare my new kernel config with the released one.
summers
93
OK - did the rebuild, and reflashed. And think I'm to the bottom of the problem.
nftables pre and post routing are know to be imcompatible with iptable_nat. Now this shows itself on in inability to do atomic replacement.
Now on 19.07 it looked like you could remove iptable_nat - but looks like it was still there.
On my respin I can definetly take iptable_nat out of the kernel. Doing this makes the difference between being able to do an atomic rule replace.
Alas though, the respind doesn't bring up the ADSL connection, so I can make an internet connection. Why isn't clear, two respin today, and its been the same - I just don't have the ADSL device.
So progress, but no solution.
OOps spoke too soon. No problem is still there - problem only occurs when doing pre and post routing ... which I guess should be a clue ...
summers
94
OK, so if you load the kernel module iptable_nat you get the errors that I'm seeing, but they are permanently there. On my machine, if I remove iptable_nat, then I get the error if the pre and post routing tables are in nf tables. So looks like there is still some of the iptable nat machinery messing with my set up.
Haven't yet been able to respin a working recompile, ADSL doesn't come up, think I know why - the firmware needed to bring up my ADSL is somewhat none standard. So need to dig into how to add it.
So in the GUI there is actually a field you can use to configure the firmware (at least it works for VDSL2/PTM) here is what it des to /etc/config/network.
option xfer_mode 'ptm'
option annex 'b'
option tone 'bv'
option line_mode 'vdsl'
option ds_snr_offset '0'
option firmware '/etc/config/vr9-B-dsl.fb7490.07.12.bin'
Whether that helps for ADSL, I do not know...
Thanks for doing all this work to debug nftables. I am eagerly awaiting a set of instructions on what's required to get nftables working, hopefully without a custom compilation, so let us know what you find!
Once we have a set of instructions that seem to work for an out-of-the-box version of OpenWrt, I'll post a new nftables QoS specific thread
1 Like
summers
97
Yes in /etc/config/network I already have from 18.06:
config dsl 'dsl'
option tone 'av'
option line_mode 'adsl'
option xfer_mode 'atm'
option firmware '/lib/firmware/lantiq-vrx200-a.bin'
option annex 'a2p'
But what I suspect is going wrong is the cryptic process:
/sbin/vdsl_cpe_control -i00_00_00_00_00_01_00_00 -n /sbin/dsl_notify.sh -f /lib/firmware/lantiq-vrx200-a.bin -l /tmp/lowlevel.cfg -M1 -T1:0x1:0x1_1:0x1:0x1
So need to check what that does on my build.
summers
98
Well I remember when i got it working faultlessly on 18.06, I had to remove many iptable packages. On 19.07 am trying to get a cleaner install. E.g. its clear for libraries and executable you only need to install nftables and that pulls in all the required, but many of the nft kernel modules need to be pulled in separately.
Anyway will suck air for a bit, digging into what is going wrong. As much as anything, respins from scratch take 24 hours, and I wonder how many rewrites my router flash will take before it gets bad bits ...
it's weird that you have so many issues but my x86 virtual machine works no problem, I don't have any of the can't use flush ruleset etc stuff. why do you think that is?
summers
100
I suspect its something that in the MIPS image gets installed in the static image, e.g. I notice ip6tables is very hard to disable. Think as much as anything whilst iptables is the primary firewall in openwrt that is where focus will be. If instead nftables were standard, these problems would be quickly solved before release.
Don't know if openwrt will ever change the default firewall, current approach fw3 uses libs that go straight into the iptable kernel, so won't work with latest kernels where iptables also uses the nftable backend. So suspect when openwrt changes to the kernels that do that will be the sensible time to swicth to nftables. Now our battle is to get nftables working reliably - so the community can then see what has to be done to support it.
What I like is the atomic replacement of the whole of nftables in one go, that is a huge advantage over iptables, that a change can be made without pulling down the old iptables. But then half the question is how to expose that in luci? I like that it shows the proposed list ruleset that you can examine before hitting a commit button.
