Hello, there seems to be considerable uncertainty in migrating to nftables in the OpenWRT 19 group of releases.
Sometimes it works, sometimes it doesn't, and when it doesn't the reasons are not obvious despite substantial efforts by experienced users.
Is there any reason to believe the migration switch from iptables to nftables will be more predictable and successful in the OpenWRT 21 release?
Has anyone tried to use nftables in OpenWRT 21?
I've not yet moved to openwrt 21.02 - but it uses a kernel where the firewall code is nftable in the kernel. This and that fw3 in openwwrt has been updated to fw. So what I suspect is happening is a total move to nftables in openwrt 21.02.
Now that fw3 has been rewritten, means I suspect this will be transparent to most users.
Now me - I always did nftables via /etc/nftables.conf - and I liked the single configuration file with its C like syntax. So when I update to 21.02 (I'll wait till full release) - I'll look to trying to keep that file. But as to if that works, have to wait and see. Then I'll update the documentation.
But my expectation, is that nftables will be far more stable in 21.02
hello @summers ,
This sounds positive. Testing will reveal full detail, but it sounds like development is moving in the right direction. I haven't migrated to 21 either and just installed 19. It's stable and working system but I'll wait to see how nftables works in 21 before upgrading.
I'm running a stratum 1 ntp time server in ntppool.org. the pool system has a way of dialing up or down the amount of traffic seen by any given server. my setup works fine at the low and medium-low settings. but anything in the mid- to high-traffic settings causes huge connection tracking tables on the router and also the ntp server. At 10,000 or 50,000 tracked udp connections, no traffic can get through. It's CPU limited. During these cases, DNS (on udp) cannot get through so it effectively stops general internet use.
I thought it was a hardware limitation and bought a better router (Ubiquiti Security Gateway) but it showed the same problem. It's a software issue.
Ideally, it would be nice to to stop connection tracking for 1 type of traffic but still provide NAT for other traffic. But configuring Iptables to stop tracking also kills NAT. Oops. Iptables won't work.
But nftables can, apparently, stop connection tracking based on port only and still provide NAT otherwise, so that's my goal with nftables.
A stratum 1 ntp! So I guess you have a high bandwidth connection to a stratum 0 server; and high bandwidth connection for stratum 2 to connect to you.
Suggests that you just let the ntp port straight through the firewall, and firewall fast in the kernel. Time is everything.
So in nftables, suggests using prerouting to make sure your packets get through with minimal delay. Think me if doing ntp stratum 1 I think about a direct connection to the internet, with no firewall. Anyway what you need to look at is very low level routing, done as soon as possible, and done with almost no delay.
Let us know how it goes!
Fw3 has not been updated. (But there is experimental not-yet supported nftables firewall4 version in the git repo)
21.02 still has the old iptables based firewall...
Interesting. I read the change from fw3 to fw, was an update - so instead just a rename.
Trying to remember when the underlying kernel was moving away from iptables. Though it was before 5.4 - I'll check up.
eik, so fw4 has been in development for many many years - but not made it to the formal release. Thats a disappointment. Suggests it stalled. Sad, but c'est la vie - I'll stick with the command line and /etc/nftables.conf then - it works well.
fw4 has only been pushed to master to date
i think there was alot of stuff blocking ease of implementation...
ifconfig > ip a ( netlink etc )... kernel > 5.x
most of that is almost behind us now... so alot cleaner to implement...
I think stratum 0 is always a physical clock, so like a GPS unit or an atomic clock of some kind. Probably what he has is a GPS receiver attached to his stratum 1 server which is then on his internet connection providing time to others.
nftables is just vastly better in every way than iptables, and it absolutely should do a better job.
I'd suggest for a stratum 1 that you use a RPi4 as the router, plug the GPS direct to it, and run the NTP server directly on it, just avoid NAT entirely.
Couldn't sleep. So checked the firewall4 git code. Correct me if wrong, but it looked to me like it's mainly the code for integrating nft command into a proper openwrt package. So in terms of firewall operation, it doesn't seem to add anything to just using the ntf directly, and writing your own nftables.conf?
If I understand it correctly fw4 is just a method of converting /etc/config/firewall commands into nft commands so that a firewall can continue to be configured by /etc/config/firewall...
if you want to use /etc/nftables.conf it should be trivial, I'm told, by just disabling the firewall and running
nft -f /etc/nftables.conf
in the /etc/rc.local
The advantage to the shift to fw4 is that it relies on nftables so one supposes that bugs in making nftables work will be considered high priority bugs as opposed to when iptables was the standard and nftables was a "you break it you bought it" kind of thing.
hello @summers,
Indeed, this is mostly all correct. I purchased a specialized gps unit that passes the high accuracy GPS Pulse Per Second (PPS) signal over the USB port to the kernel. It’s a GR-701W from Navisys. The PPS signal is typically not available over USB for most GPS devices - except this one. A few years ago ESR (Eric S. Raymond) and Gary Miller posted an idea to map out internet speeds via high accuracy timing, and Navisys built the design they suggested to enable widespread, low cost, high accuracy timing sources. GPSD describes how to feed GPS time to Chrony or ‘ntpd’.
So yes, GPS time comes from the sky, to the kernel, and Chrony monitors it carefully. Any NTP client, of any stratum, has access to this time via a public list of stratum 1 NTP servers. It is also part of the NTP Pool. The pool assigns clients to pool servers and distributed load according to (a) demand and (b) each server’s network speed setting.
Time delays in the kernel and router are of some concern, certainly. But equally important is repeatability, and stability. Chrony and ntpd use clever 2-way time transfer algorithms that account for symmetric time delays. As long as packet transfer time is repeatable and consistent, the time server and client software can get it right.
hello @dlakelan,
an RPi is a good suggestion. I started out this way and used a simple, good quality gps from Adafruit. The PPS line was fed directly into the GPIO pin and timing was spot on because hardware interrupts are fast and repeatable. I just didn’t like the mess of wires on my desk. Plus I needed the raspberry pi for another project and wanted to keep serving time. So I put the Navisys GR701W on my daily driver that’s always on. Less wires and the RPi is available for other things.
No doubt though. RPi4 is definitely a good home router solution, along with a fast USB-based Ethernet adapter.
Another person in the NTP Pool forums also suggested putting the time server directly on the public interface and using stateless iptables (with no NAT). That’s a good idea, if a spare connection were available, but this is a residential ISP. I’ve got 1 wire and 1 IP address for home internet. It’s just not practical to drop NAT.
nftables can provide good home internet and selectively provide stateless UDP on port 123 only (no conntrack on port 123). All the other traffic can remain stateful and give good internet to home as usual. This issue is a common challenge to a number of residential NTP pool servers whose connection tracking tables fill under high NTP traffic loads. There are a number of constraints, but nftables seems to solve the problem.
When I said drop Nat, I meant for the NTP connection only. Since the NTP server would run on the router and no port forwarding would be involved I imagined conntrack wouldn't get involved. But now that I think of it that's probably not right. Nftables is definitely the way to go.
net.netfilter.nf_conntrack_udp_timeout=37
net.netfilter.nf_conntrack_udp_timeout_stream=120
|| notrack
it looks like iptables has a -j NOTRACK target as well, so nftables is not the only way to turn off tracking.
@dlakelan, yes, I suspect this is accurate. conntrak needs to be addressed no matter where the NTP server is, whether router only or router + LAN-machine
@anon50098793, it's a good idea. I tried the notrack option on the router, and it worked - but it stopped all traffic on that port.
@dlakelan, from that most recent link, re-applying the ACCEPT rules may have worked, despite documentation claiming NAT and notrack is not possible together. it's not super clear but is worth revisiting
Couldn't sleep again last night, so refreshed my brain on the kernel status.of iptables.
Eventually got to the bottom of it. When iptables/xtables moved to version 1.8 the backend changed to nft. This was back in 2018, but also there was a legacy version that kept using setsockopt: legacy xtables
Now 1.8 was before openwrt 19.07 - so 19.07 is already on the 18 series iptables. But was compiled as the legacy set up (you can see this with iptables -V); so openwrt kept the old interface through to the firewall. IIRC firewall3 used the libiptc/setsockopt way into the kernel.
Now the crux for us though is what does 21.02 go towards. e.g. is it sill on iptables legacy, and when that is backed out does the npt set up work correctly. I'll let you know when the first release comes out of 21.02.
Hello @summers, I'll be interested to see your findings.