Action of 'flush ruleset' at head of 'nftables.conf' is delayed or defective resulting in error


nftables.conf begins with flush ruleset which I believe is necessary for

nft -f /etc/nftables.conf

to behave atomically and leave no security gap. (As opposed to calling nft flush rulset as a separate command.)

However, executing nft -f nftables.conf that way results in errors:

/etc/nftable.conf:3:15-24: Error: Could not process rule: File exists
	chain prerouting {
/etc/nftable.conf:7:15-25: Error: Could not process rule: File exists
	chain postrouting {
/etc/nftable.conf:9:17-43: Error: Could not process rule: No such file or directory
		oifname "br-lan" masquerade

It works fine using two commands:

nft flush ruleset
nft -f /etc/nftables.conf

Adding 128 lines of flush ruleset at the head doesn't help.

Until a couple of days ago I was using an openwrt stable release 18.06.2 with kernel 4.9.152, now it is 4.14.112. In that older version this problem did not occur even though the nft --version appears the same in both cases (see below).

Current Version:
A 04/22/2019 openwrt snapshot of
18.06.2 openwrt-brcm2708-bcm2710-rpi-3-ext4-factory.img

# uname -a 
Linux OpenWrt 4.14.112 #0 SMP Mon Apr 22 19:37:31 2019 aarch64 GNU/Linux
# nft --version
nftables v0.9.0 (Fearless Fosdick)

Can you provide the OpenWrt revision number, please. You should find it when you login to the router - in the banner.

Also, if you know the revision of the firmware it worked on, can you let us know?

Lastly, software in SNAPSHOT changes daily. So we'll also need to be sure that you installed all needed software immediately upon flashing.

Snapshot revision number

 OpenWrt SNAPSHOT, r9893-8abb505

Unfortunately I can't find the firmware version, although it was in the contents of the /boot directory belonging to that snapshot.

About the installation. The installation of nftables took place the next day. Moreover, prior to installation I removed pre-existing iptables and nftables. So it was not strictly installation only. (Details at bottom).

Can you recommend either of these following options as worthwhile?:

  • Try with another snapshot, including removing iptables/nftables and reinstalling nftables in one day
  • Use the snapshot build to compile with nftables

Method used to remove iptables and nftables and reinstall nftables

Remove all packages kmod-nf* and kmod-ip and force removal of all their dependencies.

opkg list-installed | grep -e kmod-nf -e kmod-ip | while read m x ; do opkg --force-removal-of-dependent-packages remove $m ; done

Remove all modules ip* . May require several iterations.

lsmod | grep -e "^ip" | while read m xx ;  do echo "--------- $m " ; rmmod $m ; done

Install or reinstall all packages kmod-nf* . Requires a sleep . Make sure they are ALL installed - if necessary use multiple iterations.

# opkg list | grep -e kmod-nf  | while read m xx ; do echo ":::: $m ::::" ; sleep 1 ; opkg install $m ; done
# rmmod iptable_raw
# rmmod iptable_mangle
# rmmod iptable_filter
# rmmod ip_tables
# for m in iptable_raw iptable_mangle iptable_filter ipt_REJECT ip6_tables ip_tables xt
_time xt_tcpudp xt_state xt_multiport xt_mark xt_mac xt_limit xt_conntrack xt_comment xt_TCPMSS xt_LO
G xt_CT ; do echo $m ; rmmod $m ; done
# rrmod x_tables

I did a sysupgrade to the new snapshot

OpenWrt SNAPSHOT, r9902-9424b6f

and immediately removed all package iptables and nftables packages, the installed or reinstalled every nftables package. The exact same problem occurs.

Maybe I've missed it...what version did it work on?

Following is the identification of the previous version where nft -f /etc/nftable.conf works without having to first issue nft flush ruleset in a separate command.

From the banner:

 OpenWrt 18.06.2, r7676-cddd7b4c77
root@OpenWrt:~# uname -r
root@OpenWrt:~# uname -a
Linux OpenWrt 4.9.152 #0 SMP Mon Jan 28 08:54:32 2019 aarch64 GNU/Linux
root@OpenWrt:~# nft --version
nftables v0.9.0 (Fearless Fosdick)

Notably, the nft version itself show the same identification as in the later setup with the problem.

1 Like
  • You're never clear if you used another snapshot before r9902-9424b6f, so I assume you upgraded from 18.06.2
  • As of my writing, this is still the current snapshot, so upgrading is not possible at this time
  • You haven't noted what device you're using

I see two [possibly] relevant commits if you did use a snapshot before April 24th:

Commit 488e7ccfbc was signed off by @hauke - perhaps he can give some insight.

And here all the commits noting "nft." There was a major commit in December of 2018 signed by @Ansuel - he may have insight too.

A definitive test would be to build a firmware from those particular commit points. At some point in the series of sysupgrades, nft should stop working after reboot. That commit would be the culprit of the problem.

@lleachii - Thanks much for your links, that's a big head start.

Sorry I wasn't clear. For the record where is what what worked and what didn't, and which were "factory" release, and which "upgrade":

On this stable target factory release, nft -f <file> is functioning correctly

OpenWrt 18.06.2, r7676-cddd7b4c77
Linux OpenWrt 4.9.152 #0 SMP Mon Jan 28 08:54:32 2019 aarch64 GNU/Linux

On these following snapshot releases, , nft -f <file> is not functioning correctly. The first snapshot was a "factory" installation. The second snapshot was a system upgrade of the first snapshot.

OpenWrt SNAPSHOT, r9893-8abb505
Linux OpenWrt 4.14.112 #0 SMP Mon Apr 22 19:37:31 2019 aarch64 GNU/Linux
OpenWrt SNAPSHOT, r9902-9424b6f
Linux OpenWrt 4.14.113 #0 SMP Wed Apr 24 18:02:54 2019 aarch64 GNU/Linux

I'm confused the correspondence between kernel versions and package versions.

For example your first link has a commit message "kernel: bump 4.19 to 4.19.16"

But in the particular snapshot branch is am using, the kernel starts with "4.14".

Searching for recent "kernel: bump" commits:

2019-01-17 	Stijn Tintel	kernel: bump 4.19 to 4.19.16
kernel: bump 4.19 to 4.19.16
2019-01-16 	Koen Vandeputte	kernel: bump 4.9 to 4.9.150
kernel: bump 4.9 to 4.9.150
2019-01-16 	Koen Vandeputte	kernel: bump 3.18 to 3.18.132
kernel: bump 3.18 to 3.18.132
2019-01-14 	Stijn Tintel	kernel: bump 4.14 to 4.14.93
kernel: bump 4.14 to 4.14.93

So I am guessing the snapshot is carrying several kernel versions, which for each device-project are selected via compile flags.

How about the packages? Are there separate package versions for each kernel? I think not, but if not won't that introduce incompatibilities?
A package bumped to 4.19.y might not work with kernel 4.14.x.

You are on track. Yes, different hardware are on different kernel versions. So, the buildbot properly compiles packages against those various targets/subtargets.

This is why mentioning the kernel version doesn't really help; but:

  • the OpenWrt version; and
  • the device you're using (BTW, I don't think I've asked; and you have not noted - as I recall)

It turns out that the stables release from Jan 30 2019, which runs kernel 4.9.152 and doesn't have have the bug, is supposed to also be compatible with kernel 4.14.x, although the 4.14.x kernel was not included in the release package.

The "compat with 4.14.x" info is found in this Dec 2018 commit:;a=commit;h=f5919b65d4c671fd5083838c7a445f319f9a13c8

brcm2708: add kernel 4.14 support

Just replacing 4.9.152 with the appropriate 4.14.x kernel in the stable release for 18.06.2/targets/brcm2708/bcm2710 seems like the optimal place to check first.

I've written mail to the committer of 4.14 support. I'll also contact the maintainer of 18.06.2/targets/brcm2708/bcm2710.

I changed devices and am now testing nftables the stable release version
OpenWrt 18.06.2, r7676-cddd7b4c77
Linux OpenWrt 4.14.95 #0 SMP Mon Jan 28 08:54:32 2019 x86_64 GNU/Linux

on a PC Engines apu2d4 board.

The same problem occurred, but I found an ad hoc workaround. At the top of nft file to load replace

flush ruleset


table ip nat
table ip mangle
table ip filter
flush table ip nat
flush table ip mangle
flush table ip filter

This will work both when the various tables are or are not present before loading the rules.

1 Like

You might want to drop an email about this to the openwrt-devel mailing list. You'll likely get the right set of eyes on the issue, as the commit of kernel patches is/was a pretty generic thing and there may be someone with specific interest in nftables whose eye you'll catch there (and less likely here).

1 Like

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.