nftables.conf begins with flush ruleset which I believe is necessary for
nft -f /etc/nftables.conf
to behave atomically and leave no security gap. (As opposed to calling nft flush rulset as a separate command.)
However, executing nft -f nftables.conf that way results in errors:
/etc/nftable.conf:3:15-24: Error: Could not process rule: File exists
chain prerouting {
^^^^^^^^^^
/etc/nftable.conf:7:15-25: Error: Could not process rule: File exists
chain postrouting {
^^^^^^^^^^^
/etc/nftable.conf:9:17-43: Error: Could not process rule: No such file or directory
oifname "br-lan" masquerade
^^^^^^^^^^^^^^^^^^^^^^^^^^^
It works fine using two commands:
nft flush ruleset
nft -f /etc/nftables.conf
Adding 128 lines of flush ruleset at the head doesn't help.
Until a couple of days ago I was using an openwrt stable release 18.06.2 with kernel 4.9.152, now it is 4.14.112. In that older version this problem did not occur even though the nft --version appears the same in both cases (see below).
Current Version:
A 04/22/2019 openwrt snapshot of 18.06.2 openwrt-brcm2708-bcm2710-rpi-3-ext4-factory.img
Unfortunately I can't find the firmware version, although it was in the contents of the /boot directory belonging to that snapshot.
About the installation. The installation of nftables took place the next day. Moreover, prior to installation I removed pre-existing iptables and nftables. So it was not strictly installation only. (Details at bottom).
Can you recommend either of these following options as worthwhile?:
Try with another snapshot, including removing iptables/nftables and reinstalling nftables in one day
Use the snapshot build to compile with nftables
Method used to remove iptables and nftables and reinstall nftables
Remove all packages kmod-nf* and kmod-ip and force removal of all their dependencies.
opkg list-installed | grep -e kmod-nf -e kmod-ip | while read m x ; do opkg --force-removal-of-dependent-packages remove $m ; done
Remove all modules ip* . May require several iterations.
lsmod | grep -e "^ip" | while read m xx ; do echo "--------- $m " ; rmmod $m ; done
Install or reinstall all packages kmod-nf* . Requires a sleep . Make sure they are ALL installed - if necessary use multiple iterations.
# opkg list | grep -e kmod-nf | while read m xx ; do echo ":::: $m ::::" ; sleep 1 ; opkg install $m ; done
# rmmod iptable_raw
# rmmod iptable_mangle
# rmmod iptable_filter
# rmmod ip_tables
# for m in iptable_raw iptable_mangle iptable_filter ipt_REJECT ip6_tables ip_tables xt
_time xt_tcpudp xt_state xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_LO
G xt_CT ; do echo $m ; rmmod $m ; done
# rrmod x_tables
and immediately removed all package iptables and nftables packages, the installed or reinstalled every nftables package. The exact same problem occurs.
Following is the identification of the previous version where nft -f /etc/nftable.conf works without having to first issue nft flush ruleset in a separate command.
From the banner:
OpenWrt 18.06.2, r7676-cddd7b4c77
root@OpenWrt:~# uname -r
4.9.152
root@OpenWrt:~# uname -a
Linux OpenWrt 4.9.152 #0 SMP Mon Jan 28 08:54:32 2019 aarch64 GNU/Linux
root@OpenWrt:~# nft --version
nftables v0.9.0 (Fearless Fosdick)
root@OpenWrt:~#
Notably, the nft version itself show the same identification as in the later setup with the problem.
A definitive test would be to build a firmware from those particular commit points. At some point in the series of sysupgrades, nft should stop working after reboot. That commit would be the culprit of the problem.
@lleachii - Thanks much for your links, that's a big head start.
Sorry I wasn't clear. For the record where is what what worked and what didn't, and which were "factory" release, and which "upgrade":
On this stable target factory release, nft -f <file> is functioning correctly
OpenWrt 18.06.2, r7676-cddd7b4c77
Linux OpenWrt 4.9.152 #0 SMP Mon Jan 28 08:54:32 2019 aarch64 GNU/Linux
On these following snapshot releases, , nft -f <file> is not functioning correctly. The first snapshot was a "factory" installation. The second snapshot was a system upgrade of the first snapshot.
2019-01-17 Stijn Tintel kernel: bump 4.19 to 4.19.16
kernel: bump 4.19 to 4.19.16
2019-01-16 Koen Vandeputte kernel: bump 4.9 to 4.9.150
kernel: bump 4.9 to 4.9.150
2019-01-16 Koen Vandeputte kernel: bump 3.18 to 3.18.132
kernel: bump 3.18 to 3.18.132
2019-01-14 Stijn Tintel kernel: bump 4.14 to 4.14.93
kernel: bump 4.14 to 4.14.93
...
So I am guessing the snapshot is carrying several kernel versions, which for each device-project are selected via compile flags.
How about the packages? Are there separate package versions for each kernel? I think not, but if not won't that introduce incompatibilities?
A package bumped to 4.19.y might not work with kernel 4.14.x.
You are on track. Yes, different hardware are on different kernel versions. So, the buildbot properly compiles packages against those various targets/subtargets.
This is why mentioning the kernel version doesn't really help; but:
the OpenWrt version; and
the device you're using (BTW, I don't think I've asked; and you have not noted - as I recall)
Just replacing 4.9.152 with the appropriate 4.14.x kernel in the stable release for 18.06.2/targets/brcm2708/bcm2710 seems like the optimal place to check first.
I've written mail to the committer of 4.14 support. I'll also contact the maintainer of 18.06.2/targets/brcm2708/bcm2710.
You might want to drop an email about this to the openwrt-devel mailing list. You'll likely get the right set of eyes on the issue, as the commit of kernel patches is/was a pretty generic thing and there may be someone with specific interest in nftables whose eye you'll catch there (and less likely here).