Nftables compatibility and performace issues in OpenWrt 23.05.2 (and 22.x.x)

helloyello · January 30, 2024, 4:16am

Hi,

ATM I'm still stuck with OpenWrt 21.x.x on a D-Link DIR-860L B1 and since I noticed some developments with the possibility to add a custom firewall in OpenWrt 23.x.x I considered to switch to the latter.
Reference:

Now, since I'm very comfortable with iptables, have more than 2 decades of system/network admin experience (that's accustomed with laggy remote shells/terminals, rather dumb editors like vi in which you don't really look to produce fancy accolades indentations ... and happy to parse, understand and resolve network & firewall syntax issues fast, advocating to leave the utterly inefficient love poems syntax to the poets) and not a developer/coder like oshenders - see Reflections:

I tried to use iptables-nft for my custom rules (script), knowing that it should be compatible (nftables documentation) - Section: using the nf_tables compat backend:
https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables
and failed pretty bad at it in OpenWrt 23.05.2.

I didn't want to mess up the main OpenWrt 21.x.x on a D-Link DIR-860L B1 and used a spare TP-Link TL-WR841N v13 loaded with OpenWrt 23.05.2 as guinea pig. Hooked it on WAN to the main D-Link DIR-860L B1 in a double-NAT fashion - D-Link DIR-860L B1 is on 192.168.1.0 and TP-Link TL-WR841N v13 on 192.168.2.0
To recap, all the tests were done on a TP-Link TL-WR841N v13 loaded with OpenWrt 23.05.2 and configured in a Routed AP fashion.

In this forum I learned that I would need some additional packages to make iptables-nft work and installed the following:

opkg install iptables-nft
opkg install ip6tables-nft
opkg install kmod-ipt-conntrack
opkg install kmod-ipt-nat

Before "translating" my old OpenWrt 21.x.x iptables custom ruleset into iptables-nft, I started to test some rules in a ssh console on the router. For example, I defined 2 generic rules to allow the traffic on tcp port 9999 (Kodi Remote - KORE App) originating from my phone over WiFi to access a station connected to the LAN (Kodi Player) - note that the router was configured in Routed AP mode (LAN/WiFi split and forward traffic dropped inLuCi-Firewall Settings).
The rules:

/usr/sbin/iptables-nft -I FORWARD -i phy0-ap0 -o br-lan -p tcp -s 0/0 --destination-port 9999 -j ACCEPT
/usr/sbin/iptables-nft -I FORWARD -i br-lan -o phy0-ap0 -p tcp -d 0/0 --source-port 9999 -j ACCEPT

Well, it didn't work and I started to investigate. Launched tcpdump on both OpenWRT and the Kodi station (LAN) and learned that the packets were coming over WiFi into the router, were forwarded to the LAN station, the LAN station replied and the replies were dumped by the router (didn't hit my second rule). Cool!
Furthermore, found out that iptables-nft creates its own table "ip filter" and doesn't really add/delete rules in/from the OpenWRT defined table "inet fw4". Again, just wonderful.
Now, I could see the packets (counters) coming from WiFi hitting the first rule in the ip filter table, but the second rule counters were all on 0. My guess is that the replies were dropped in the rules from within the table inet fw4.
I didn't investigate further because my eyes were hurting, zig-zaging through that rather uninspired unindented output of "nft list table inet fw4/ip filter & nft list chain * "
Also important, with just a few (less than a dozen) of these iptables-nft rules, the throughput of the router dropped from 100Mbps to ~12Mbps (both uplink/downlink) - noticed it while performing a speed test (speedtest.net) through it.

In some posts and on github (OpenWRT related) I found some additional packages that I also installed (never know what I might need in my future firewalls), apparently without any positive results.
List:

opkg install iptables-mod-filter
opkg install iptables-mod-conntrack-extra
opkg install kmod-ipt-ipset
opkg install iptables-mod-nfqueue
opkg install kmod-ipt-nat6
opkg install kmod-ipt-ipopt
opkg install kmod-ipt-compat-xtables
opkg install kmod-ipt-condition
opkg install kmod-ipt-extra
opkg install kmod-ipt-iface
opkg install kmod-ipt-ipmark
opkg install kmod-ipt-iprange
opkg install kmod-ipt-ipv4options
opkg install kmod-ipt-nat-extra
opkg install kmod-ipt-physdev
opkg install kmod-ipt-proto
opkg install kmod-ipt-raw
opkg install kmod-ipt-raw6
opkg install kmod-ipt-rpfilter
opkg install kmod-ipt-socket
opkg install kmod-ipt-sysrq
opkg install kmod-ipt-tproxy

Added a little masochism to my approach and started to consider that if I can't beat nftables, well, let's embrace that abomination. I could always use iptables-translate to please OpenWrt
iptables-translate doesn't always output 100% nft compatible/native syntax and some fine tuning is required on complex iptables statements (had a few in a ~ 200 lines ruleset).
But then, another proof that the folks behind nftables do not appear to have much sys/net admin field experience given how inconsistent and incompatible with shells the syntax is (brackets, & signs, then pipes and then curly brackets). Had to escape a lot for pleasing the ash shell - a small snippet from a block of traffic integrity check I always use (not just drop inconsistent packets but also inconsistent traffic):

# brackets and pipes for flags series
/usr/sbin/nft insert rule inet fw4 input tcp flags \& \(fin\|syn\|rst\|psh\|ack\|urg\) == fin\|syn\|rst\|psh\|ack\|urg counter drop
/usr/sbin/nft insert rule inet fw4 input tcp flags \& \(fin\|syn\|rst\|psh\|ack\|urg\) == fin\|psh\|urg counter drop
/usr/sbin/nft insert rule inet fw4 input tcp flags \& \(fin\|syn\|rst\|psh\|ack\|urg\) == fin\|syn\|rst\|ack\|urg counter drop
# and then comma separation and curly brackets for port series ..wtf
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" ip protocol tcp ip saddr 192.168.10.101 tcp dport { 111,2049,32765,32766,32767,32768,32769} counter accept

Finally, all works (apparently) as expected with my nft ruleset and the custom script triggered from within the firewall conf file, only issue is that by adding a bunch of these nft rules (around 50 from about a total of 200) I loose around 50% of the throughput - speedtest reports ~ 52Mbps.
Commented most of the 50 rules, left with a dozen and I still lose around 30% of the throughput.
Worth noting that on this router with OpenWrt 21.x.x and the full iptables ruleset (200 lines) I lose around 3-5Mbps (speedtest result ~96Mbps).

Now, my only question is simple:

while OpenWrt 22.x.x looks useless, there is OpenWrt 23.x.x as the latest and greatest with all these (dis) functionalities embedded, would it be possible to still maintain (security patches only) OpenWrt 21.x.x - the "Last Action Hero" for a while?
I'd be really thankful for such an effort and kindness and I really mean it. Reading many of the posts here I started to become confident that you'll also make a lot of other users happy while still supporting iptables.

Thanks!

flygarn12 · January 30, 2024, 5:14am

How much free ram memory do the device have during this test.

What is the speed drop with a default setup?

There are a lot of fw4 running out there and no one has really noticed a speed drop yet as far as I can remember.

And if you remove all those ip table packages you installed, does that make any difference?

DBAA · January 30, 2024, 5:49am

Take some time to learn nftables and the nft command then write new rules that take advantage of what nftables offers to do what you need. Shoehorning iptables rules in will leave you with a suboptimal configuration both in complexity and performance.

helloyello · January 30, 2024, 6:01am

I performed two tests now, one with the default OpenWRT firewall rules and no additional custom rules.

OpenWrt default firewall - synflood & drop invalid packets enabled

- speedtest result 94 Mbps

- htop
  CPU[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%] Tasks: 25, 0 thr, 33 kthr; 0 running
  Mem[||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||                    21.6M/55.9M] Load average: 0.27 0.20 0.09
  Swp[                                                                                                          0K/0K] Uptime: 1 day, 00:10:33

- top
Mem: 40856K used, 16416K free, 124K shrd, 0K buff, 15260K cached
CPU:   0% usr   0% sys   0% nic   0% idle   0% io   0% irq  99% sirq
Load average: 0.38 0.25 0.11 2/57 15951

OpenWrt default firewall - synflood & drop invalid packets disabled and custom rules added

- speedtest result 52 Mbps
- htop
  CPU[|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||100.0%] Tasks: 25, 0 thr, 33 kthr; 0 running
  Mem[||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||                      20.5M/55.9M] Load average: 0.39 0.12 0.03
  Swp[                                                                                                          0K/0K] Uptime: 1 day, 00:06:59

- top
Mem: 39180K used, 18092K free, 124K shrd, 0K buff, 15260K cached
CPU:   0% usr   0% sys   0% nic   0% idle   0% io   0% irq  99% sirq
Load average: 0.43 0.19 0.07 2/58 15795

Interesting, CPU was at 100% loaded handling softIRQs in both tests (in htop I had the magenta bar maxed out), but there's an additional ~1MB of extra RAM used without my custom rules...
But then I noticed some weird behaviors in these older nftables performance test:

https://www.semanticscholar.org/paper/LATENCY-AND-THROUGHPUT-COMPARISON-BETWEEN-IPTABLES-Ding/fae47cbc75e0bfc2cc1bee43ba178d4658b6789f

Sorry I forgot to mention, indeed I removed all the additional iptables-nft and iptables packages when I switched to native nft - actually I resetted the router and started clean. (should have been obvious ... )

helloyello · January 30, 2024, 6:11am

Thanks for the advice. As you may have well noticed I already wasted some time, the whole weekend actually, with this nftables abomination and I must admit, I'm unimpressed.

Apparently nftables doesn't really add to performance gain nor ease of use/simplicity, not to mention the inefficient and inconsistent syntax - writing love poems in curly brackets... in vi. Could be the ultimate height in sado-masochism

The single real advantage of nftables I learned about over iptables is that adding a rule doesn't flush& reload the whole set ( the previous ones) , as it does in iptables,. But nftables implementation in OpenWrt (fw4) doesn't make use of this improvement/functionality, as it flushes everything on every modification (reload).

flygarn12 · January 30, 2024, 7:47am

How many spectacular super rules do you actually have that must be written in nftables code instead of uci code or in luci?
At least do the normal rules in uci code and if anything super special rule is left than make only those rules in nft code.

Even with your cv above. Maybe you are not so good at writing nftable code as you think you are?

It is no problem having 50 or 100 or more rules (i am running that level myself) without any performance drop when letting the system write the nft rules based on what you want to do in more simple text.
If the fault was in OpenWrt we would have seen it here in the forum years ago, but it is your manual nft config that have the problem.

helloyello · January 31, 2024, 4:28am

I genuinely believed you were interested in what I exposed in my original post and looking to add value to the discussion, hope you still are. In this respect, please do not spread false information and false personal attacks/accusations, read again more carefully my original post instead.

Just to clear your confusion, I reiterate: I do have around 200 iptables rules, around half of which are simple traffic integrity checks, flood protections and co., I provided already a snippet from these in my original post. The other half are normal interconnection allowances (local services) between Wi-Fi, LAN stations and some forward (to the internet) filters, the latter mainly taming Redmond and some smart devices that want to call home.

The rather limited Luci clicky-clicky is OK for a bunch of simple traffic/port fw rules - 5-10, because otherwise you'll start to lose the overview in that web representation. Furthermore, the documentation of the fw3 & fw4 is not really complete and that put me off (historically speaking) using it. I didn't know you can negate expressions, use series of IP's, not even knowing you could use CIDR notation. All these I had to find on my own by testing (trial & error) and in some forum/github comments (OpenWrt devs sharing knowledge ). So, in conclusion, I didn't touch LuCi that much and minded my own business with iptables custom rules and was happy with it (the flexibility).

Now, I don't recall publishing my "cv", I just said that I have more than two decades of sysadmin&networking experience, more into integration/sustaining capacities, actually in 2028 it'll be 30 years. I started in '98, working at a local ISP where I had to HW restart a dumb Windows NT (RADIUS & billing) almost every second day during the night - that's waking up, driving at the office and pushing the magic button. It's then I discovered FreeBSD and Slackware, gradually substituting that Windows NT mess. I lost BSD along the way but still a Slacker to this very day. iptables is not common in enterprise env., at least it wasn't some 5-7 years ago, but only dedicated (and expensive) FW HW from CISCO/Juniper and this kind of equipment I had to design & interconnect & configure. Privately and in some SOHO scenarios I intensively used iptables, quite a learning curve given that there wasn't much info and a lot of mistakes I had to learn from. Finally, it wasn't my intention to show off, rather point out that I know my stuff in networking and firewalling, at least for IPv4, and "warn" to not waste time with useless off-topic arguments.
And no, I'm not at all good at writing anything related to nftables, I said that I'm using iptables-transalte and just fine tuning the erroneous outputs.
I can give you some examples of the "unfinished" (rather beta stage) nftables toolset B$:

#This is the rubbish iptables-translate produces:
- original iptables rule:
/usr/sbin/iptables -I FORWARD -i phy0-ap0 -s 192.168.20.100,192.168.20.101 -p udp -d 0/0 ! --destination-port 123 -j DROP
- parsed through iptables-translate (note the split and the total miss of the negation):
nft insert rule ip filter FORWARD iifname "phy0-ap0" ip saddr 192.168.20.100 udp dport 123 counter drop
nft insert rule ip filter FORWARD iifname "phy0-ap0" ip saddr 192.168.20.101 udp dport 123 counter drop

- original iptables rule:
/usr/sbin/iptables -I FORWARD -i eth0.2 -s 0/0 -p udp -d 192.168.20.100,192.168.20.101 ! --source-port 123 -j DROP
- parsed through iptables-translate (note the split, this time it recognized the negation, why?)
nft insert rule ip filter FORWARD iifname "eth0.2" ip daddr 192.168.20.100 udp sport != 123 counter drop
insert rule ip filter FORWARD iifname "eth0.2" ip daddr 192.168.20.101 udp sport != 123 counter drop

And the manually corrected nft rules I put in my custom firewall:

# Smart Plugs - NTP - allow for stats!
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" ip saddr { 192.168.20.100, 192.168.30.101 } udp dport != 123 counter drop
/usr/sbin/nft insert rule inet fw4 forward iifname "eth0.2" ip daddr { 192.168.20.100, 192.168.30.101 } udp sport != 123 counter drop

It looks I'm better than the beta tools the nftables team provided....

On your last points. Indeed it wasn't any problem (performance impact) to have a set of 200 iptables rules in OpenWRT 21.x and that's why I used&kept them. Not the same apparently with nftables.
And no, I don't believe you would have noticed "If the fault was in OpenWrt", first because a Routed AP configuration like mine (again I guess you didn't pay attention to the details in my OP) is not really common (it's actually quite complicated if you also drop the forward between networks (isolate them) - like I do) and there were no years for OpenWrt users to play around with custom firewalls based on nftables - that's again false information.
The custom script feature was dropped in OpenWrt 22.x, there were some discussions (and desperate cries) here on how to hack the system and still use it by modifying some init files and it got "officially" reintroduced recently (some months ago) in OpenWrt 23.x.x (not in LuCi, but in the fw4 documentation). And this is what triggered me to give it a try.

Back to objectivity, in my OP I described that I'm using this rather old and lazy TP-Link TL-WR841N v13 as a test bed, getting accustomed and confident enough to switch my main router to OpenWrt 23.x.x.
On this TP-Link TL-WR841N v13 even with the "standard" traffic rules OpenWrt 23.x.x comes with and no custom firewall I loose around 5% throughput - that's 92Mpbs instead of 96-98Mpbs with OpenWrt 21.x.x
To be more precise, I disabled everything related to ipv6 in the networking config, set the forward between local networks on drop and also canceled out a bunch of the "standard" FW rues:

 /etc/init.d/firewall restart
Section @rule[2] (Allow-IGMP) is disabled, ignoring section
Section @rule[3] (Allow-DHCPv6) is disabled, ignoring section
Section @rule[4] (Allow-MLD) is disabled, ignoring section
Section @rule[5] (Allow-ICMPv6-Input) is disabled, ignoring section
Section @rule[6] (Allow-ICMPv6-Forward) is disabled, ignoring section
Section @rule[7] (Allow-IPSec-ESP) is disabled, ignoring section
Section @rule[8] (Allow-ISAKMP) is disabled, ignoring section

Then, I did shrink and simplified my custom firewall to around 25 (native) nft rules, the very basic I need for my network and services to function in the Routed AP fashion and noticed a 10% throughput drop - speedtest got around 82-82 Mbps in both directions.
Here is the custom ruleset as exemplification:

#!/bin/sh
# Firewall nft scrip for OpenWrt

# LAN local services
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol udp udp dport { 53,44001,44002,44003} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol udp udp sport { 53,44001,44002,44003} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" tcp dport 44000 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" tcp sport 44000 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" tcp dport 44999 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" tcp sport 44999 counter accept
# WiFi local services
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" ip protocol udp udp dport { 53,44001,44002,44003} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" ip protocol udp udp sport { 53,44001,44002,44003} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" tcp dport 44000 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" tcp sport 44000 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" tcp dport 44000 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" tcp sport 44000 counter accept
# tvheadend LAN & WiFi
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol tcp tcp dport { 9981,9982} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol tcp tcp sport { 9981,9982} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol udp udp dport { 46444,1900,57552} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "br-lan" ip protocol udp udp sport { 46444,1900,57552} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" ip protocol tcp tcp dport { 9981,9982} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" ip protocol tcp tcp sport { 9981,9982} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" ip protocol udp udp dport { 46444,1900,57552} counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" ip protocol udp udp sport { 46444,1900,57552} counter accept
# Kodi Web & KORE
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" tcp dport 9999 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" tcp sport 9999 counter accept
# Smart Plugs -  remote control(script)
/usr/sbin/nft insert rule inet fw4 forward iifname "br-lan" oifname "phy0-ap0" ip saddr 192.168.10.50 tcp dport 80 counter accept
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" oifname "br-lan" ip daddr 192.168.10.50 tcp sport 80 counter accept
# Smart Plugs - NTP - allow for stats!
/usr/sbin/nft insert rule inet fw4 forward iifname "phy0-ap0" ip saddr { 192.168.20.100, 192.168.30.101 } udp dport != 123 counter drop
/usr/sbin/nft insert rule inet fw4 forward iifname "eth0.2" ip daddr { 192.168.20.100, 192.168.30.101 } udp sport != 123 counter drop

Again, all these just to clear out your confusion and counter your false statements.

Meanwhile I spent some time playing around with nftables, getting used with the impossible syntax, and learned that deleting a rule is quite a challenge:
https://wiki.nftables.org/wiki-nftables/index.php/Simple_rule_management#Removing_rules
I added a rule in the forward chain and had to inspect a whole page of that dumb zig-zag nft output to pin point it (handle 8417) and delete it with:

nft -a list chain inet fw4 forward
nft delete rule inet fw4 forward handle 8417

.. that's it for me I guess.

I complained objectively about the syntax and I still don't understand why "the wheel was reinvented" with some user/shell (totally) unfriendly, inconsistent and uninspired syntax instead of keeping the well known. well documented iptables syntax.
But with the latest experience I'm confident now that this nftables pile of ... unfinished work is something I don't really have time to beta test. Feel sorry for OpenWrt that it had adopted it, quite hastily I must add. Must be that the nftables devs are rubbing their hands in happiness, having such a great tribe of beta testers now

Will stick to OpenWrt 21.x.x and considering to move the routing on a SH thin client with a full fledged Linux. Thanks for all the effort (been using OpenWRT since 14.x/15.x).
Will set this thread on resolved. Still, I would be happy if my question from the OP would receive a positive answer from the devs.

bluewavenet · January 31, 2024, 4:16pm

I hear your frustration and felt it myself initially.
Then I was forced to migrate a quite complex package from iptables to nftables and started the task with a great deal of trepidation and foreboding, and a complete lack of confidence in the "new way of doing things".

My first attempts at translating gave mediocre results at best. Then the penny dropped.

Iptables has only a small subset of the abilities of nftables.
The "penny dropping" was my realisation that simple translation is only a small part of what is required when migrating an application using nftables.

The package iptables-nft is very simply a shim that if you are lucky will allow packages designed with iptables to run with nftables in a form of iptables compatibility. A stop-gap if you will, while awaiting package migration.

That compatibility involves iptables-nft creating a pseudo iptables ruleset within the potentially much larger nftables ruleset. And yes, performance generally suffers.

In nftables you can define as many custom tables as you like (compared with iptables with fixed tables).

Your custom tables can have as many chains as you like with higher or lower priority in the ruleset compared with other tables/chains.

The OpenWrt firewall, "fw4", has only a single table with many chains. So custom tables/chains can be higher, the same or lower priority than fw4 chains/rules, depending on your use case.

Fw4 is a static nftables configuration tool designed to use a single firewall config file and it works well for configuring a simple router, but its ruleset is somewhat convoluted and can rapidly get very complicated and therefore can be inefficient when people like you or me get started

Building your own ruleset independent of fw4 and with the required priorities is the way forward unless you just want a few extra and simple uci compatible rules.

Restricting yourself to hardware compatible with generic Linux distros is possibly one way forward, but most have also changed to nftables, with iptables-nft installed by default so in effect you would be no further forward.

If you want to look at the package I migrated, you can see it here:

It is a dynamic firewall in its own right and has to react to traffic down to a user level and dynamically adjust its ruleset as required.

This might be much more than you require but will almost certainly give you a better understanding of how nftables can be used.

With hindsight, if I was migrating it now, I would code it a bit differently, but it was a learning curve and the end result works well.

You are welcome to look at the code and open an issue there if you want to discuss any details.

Lynx · January 31, 2024, 4:34pm

Whilst desirous to respect my elders, I confess to being glad that I completely bypassed iptables and so didn't have to contend with the transition that you dinosaurs have had to face.

As we get older we: begin to lose our faculties, sometimes get grumpy; find adapting to change harder; and our grasp on reality weakens. This is poetically and humorously expressed here:

Remember your Creator in the days of your youth, before the days of trouble come and the years approach when you will say, “I find no pleasure in them”— before the sun and the light and the moon and the stars grow dark, and the clouds return after the rain; when the keepers of the house tremble, and the strong men stoop, when the grinders cease because they are few, and those looking through the windows grow dim; when the doors to the street are closed and the sound of grinding fades; when people rise up at the sound of birds, but all their songs grow faint; when people are afraid of heights and of dangers in the streets; when the almond tree blossoms and the grasshopper drags itself along and desire no longer is stirred. Then people go to their eternal home and mourners go about the streets.
-Ecclesiastes 12:1-5

I think it is wise to acknowledge our advancement toward old age and infirmity. Happens to everyone. This is the human condition.

bluewavenet · January 31, 2024, 4:52pm

tievolu · January 31, 2024, 5:44pm

I'm another iptablesaurus, having used iptables for years and with about 600 iptables/ipset commands generated by a large script, mostly related to traffic prioritization with DSCP tags etc.

I did try iptables-nft initially, hoping for a swift and easy way forward, but it became clear very quickly it wasn't going to work properly. As others have said, the only real solution is to dive in and learn nftables.

I put this off for ages and completely skipped Openwrt 22.03 (sticking to 21.02) to avoid it. But there were other improvements in 23.05 that I really wanted, so I had to bite the bullet. Like all these things it's horrid at first and if you're anything like me you'll feel like a complete idiot at times, but it gets better quickly, and iptables-translate can be a very useful learning aid to point you in the right direction.

Having now spent some time with nftables I can certainly see that it is extremely versatile. I was able to do everything I needed to, and in many cases I could (eventually) do it more efficiently that I could with iptables. IMO the syntax is also much nicer to read once you get used to it.

On the downside I've encountered some performance issues of my own, particularly with large sets, but nothing that was a deal breaker.

Regarding the need to constantly escape characters in nft commands, this irked me as well and the best solution is to bypass the shell by putting all your rules in an nft script, and then pass that into the nft command:

nft -f [script file]

With this approach I think the only escaping that's required is inside rule comments.

efahl · January 31, 2024, 6:35pm

Just curious, do you have an example of what needs to be escaped? I've just always used double quotes around my comments, so maybe that avoids this?

tievolu · January 31, 2024, 6:47pm

I can't look at my rules right now but IIRC I think I have brackets and greater than signs escaped.

Maybe that's a hangover from my previous incantations in the ash shell though and I don't actually need to do that.

EDIT: I just checked and it does indeed look like those characters don't need to be escaped

helloyello · February 1, 2024, 1:05am

Some interesting statements in your reply - blame me for cherry picking, but I couldn't stand my hand not replying to them (considering them false assumptions).

You mean the new stupid, overcomplicated, incomplete and usability (user/shell) unfriendly way of doing things ? If yes, I feel you then.

Never heard of that, not even the nftables developers are expressing such statements, but more highlighting the improvements and efficiency in the kernel ABI and userspace API. Although, given my concrete experience now with the performance impact, I'm not sure about all these "improvements".

Right, and that will help in what marginal case? The average Joe, not specialized (not a guru) in networking, with a simple (and nowadays crucial/necessary) firewall requirement will really appreciate that.

Very good, in iptables you were not able to mark packets and then treat the flows (which are the main focus/concern) accordingly with tc ... specialized toot for specialized purpose. This looks to me like a solution for an nonexistent problem.

Well, I did that and it didn't work. Coming back to the many tables and no f... clue where the packets were filtered (which rules from which table were hit).

I don't know about "most" as I'm a Slackware user and Slackware doesn't teach (impose) the user what to do/use/choose. Never did, never will. Actually the firewall script provided is not even executable and empty.
Nor do I care for the most/popular distros, as I don't really regard them anymore as Linuxes (obviously they are, at least based on standard Linux kernel, libs and tools), but a mess of Redmond-like automations/helper scripts/abstractions/apt-get/apt-dont-get that I have to fight and tame in order to achieve simplicity and flexibility. I don't necessarily advocate Slackware either, it's a royal PITA if you don't know Linux (because it's Linux (tools) in its "purest" form - design philosophy is on Wikipedia). Nevertheless, I understand/appreciate/respect that some popular distros like Debilan/Ubuntu are easier to use especially for devs looking for a simple (to update/maintain) build environment.

Having said all that, I understand that there are some improvements/extensions in the architecture and usability compared with iptables and it's expected to be like that given it's an evolution. Well, I still regard it as a regression in all its extent.
But then, I see this nftables thing mainly as incomplete work and therefore I cannot appreciate/respect it, not impressed. Furthermore, in my objective and concrete analysis (and shocking experience) I was criticizing mainly the syntax, which I regard as inconsistent (brackets, curly brackets, pipes and commas for series separations, etc.), counterintuitive to some extent (no clear highlighting of chains networks/IPs, switches for easy inspection/parsing and understanding of rules blocks), inefficient (too many words - poetry?), that unformatted zig-zag output of nft list is just plain f... stupid and finally the hurdle to delete a rule was the last drop for me (might need some AI help to parse the ruleset and identify the targeted line....).

I asked myself why the properly documented/well known/well designed iptables syntax was not adopted (extended with new features/flags/directives too - borrow the already available source code) and I fear that the nftables devs, out of exuberance/lack of (sysadmin/networking) field experience, dev (coding) minded approach and a potential lack of proper project/architectural management (inconsistency in the syntax development) produced the actual abomination. Might be exciting to adopt this abomination for other devs (coding minded) or script kiddies, having plenty of time to kill with curly brackets in advanced editors, but not for normal("mortal") sys/net admins. Nothing to feel enlightened about the repulsiveness and ambiguity of the actual usability,
And there's another worrisome point - I said I had my own pretty long learning curve with iptables, given it wasn't properly documented, not much discussions and forums (decades ago), but it was an easier life back then. There were not that many automated attacks/scans/botnets/exploits/ IoT spyware...etc at that time, so a mistake was "tolerated", nowadays is more "catastrophical". Now, in these very "nasty" times you come with some utterly dumb and overly complicated pile of ... unfinished work and impose it over the poor little f**ker/user, expect her/him to learn it fast and use it correctly. Get real, would you?

One last point, I'd focus on the scope of OpenWrt as a customized, limited in flexibility (not a full fledged Linux) and dedicated OS for routing, designed for running on mainly very limited HW capabilities and used by "consumer-level" average Joe, who's neither a dev nor a script kiddie. This average Joe, one might argue, could use the LuCi provided forms for traffic filtering (or even write uci blocks in /etc/config/firewall) and I agree with that to some extent, at least for a few simple port forwarding and filtering rules. But once you have a dozen of those, you loose oversight in that UI/uci representation and it becomes painful/time consuming to clicky-clicky in LuCi or edit the firewall file and scroll/add/delete rules ... in vi.

While iptables is around (apparently developed) and not EOL, I prefer to stick with it. Thank you very much.

I rest my case.

DBAA · February 1, 2024, 5:17am

Linux added nftables a decade ago with Linux kernel version 3.13. Nothing hasty or rushed about it. Developed by the same team that did iptables but with an eye to the future - performance and reduction in code complexity/duplication among the goals.

Slagging nftables because you are struggling with it is poor form.

helloyello · February 1, 2024, 7:36am

Added as work in progress and not imposed, and you should do some more research about the "same" team.
Not slagging, but objectively criticizing it, not struggling but rather avoiding an unnecessary PITA affecting both my efficiency and effectiveness.
And in the end caring (maybe too much) and not writing superficial rubbish and personal attacks like you do over here.

Lynx · February 1, 2024, 8:22am

If nftables is really so bad then how can we make sense of it having replaced iptables as the default framework (and not just in respect of OpenWrt)? Hype can only go so far, right?

helloyello · February 1, 2024, 8:35am

Where and since when did it replace iptables? If you mean Debilan and Ubuntu, well, those were always "the new kids on the block" showing off with the latest and greatest, soon competing with Redmond in feature richness and ease of use (my opinion).
I wouldn't compare OpenWrt with those distros, just no sense at all.

Don't see any hype but a rush decision and I was wondering myself already 1 year ago why OpenWrt enforced this unfinished mess in at that time some release candidate - see:

But then again, I closed this thread myself and am tired of arguing objectively with subjectively minded folks on subjective impressions and beliefs. I was originally addressing the devs with a simple question/request.

Ramon · February 2, 2024, 2:35pm

How is this a simple question /request?

helloyello · February 2, 2024, 7:52pm

Just noticed you're not a developer but an OpenWrt user since 2019, thus not sure how you represent the developers.
And to clear out your confusion, this is a quote from my OP (simplified, code base for OpenWrt 21.x.x. is still available and security patches could be applied/backported):