21.02 > 22.03 - iptables custom rules migration

tievolu · July 11, 2022, 12:50pm

Apologies if this has been covered elsewhere. I did search the forum and the web but couldn't find answers that address my specific situation.

I'm currently running 21.02 and have lots of custom iptables firewall rules, mostly for tagging connections/packets with DSCP marks, but sometimes in non-trivial ways devised over many months. Obviously at some point I will need to upgrade to 22.03, but I'm concerned about the switch from iptables to nftables and how much pain that's going to cause me.

Along with many of the core iptables features I use:

packet marks
connection marks (with masks, for storing and restoring DSCP marks)
hash limits (for flow rate measurements based on ports and IPs)
ipsets (static / dynamic / populated by dnsmasq)

I use these iptables packages/modules:

iptables-mod-conntrack-extra
iptables-mod-extra
iptables-mod-hashlimit
iptables-mod-ipopt
ipset
kmod-ipt-ipset

I've read that iptables-nft is "fully compatible" with iptables, but does that mean all of the above iptables features are still available and working in 22.03? i.e. can I just upgrade to 22.03, install these packages along with iptables-nft, and expect everything to work? I know that ipsets don't exist anymore with nftables, for example.

trendy · July 11, 2022, 2:42pm

Whatever exists in uci firewall configuration will be migrated automatically. Custom iptables scripts in /etc/firewall.user however won't be. That being said, you might want to start migrating your custom scripts to fw3, for example marks are supported.

tievolu · July 19, 2022, 9:22am

Thanks for the response, but I'm still not sure where I stand. I know that at some point I'm going to have to bite the bullet and learn nftables to the same extent that I know iptables, but that's not going to be trivial.

I guess the question boils down to whether iptables-nft really is "fully compatible" with iptables, i.e. can I install those iptables modules? Will they work? Will ipsets still work with iptables-nft?

_bernd · July 19, 2022, 9:51am

I have not yet checked where custom nft rules are setup but you can translate your iptables rules to nft with https://wiki.nftables.org/wiki-nftables/index.php/Moving_from_iptables_to_nftables

hnyman · July 19, 2022, 10:08am

Note the difference in compatibility in the iptables/nftables layer and in the fw3/fw4 uci firewall layer.

Eventhough iptables-nft would be fully compatible, you need lots of work, as the table/chain/rule structure created by the uci fw4 firewall is different than the one from fw3, so likely your iptables commands would need modifications so that they would apply to the corrent chains.

tievolu · July 19, 2022, 10:19am

Ok, that's good to know. For most of my stuff I create my own chains that hang off the base prerouting/output chains, so changes to OpenWrt's chains shouldn't affect me too much, although there are definitely a couple of scripts of mine that will need fixing.

I'm also very interested in nftables' ingress chain. Presumably this chain could be used to manipulate/restore DSCP marks on download packets before they're seen by SQM?

Also, I found this mail on the nft mailing list which indicates that iptables-nft can use ipsets, which is encouraging:

Dopam-IT_1987 · July 19, 2022, 11:44am

Why don't you convert your iptables rules to nftables?

tievolu · July 19, 2022, 12:09pm

Ultimately that's what I'll do.

But it's not quite that simple. I have scripts that set up the rules in a way that's non-trivial, but makes the ruleset much easier to modify and maintain - one of these scripts generates about 300 iptables rules. I can't convert those scripts without understanding the nftables syntax properly. Yes, I could export my current set of iptables rules created by the scripts and convert them to nftables, but I would effectively be throwing the scripts away and I don't want to do that.

I'm also still not sure whether nftables (or even iptables-nft) will support all the iptables functionality in currently using (e.g. hashlimits).

When I upgrade to 22.03 I plan to convert the iptables rules to nftables and use the results to help me learn how nftables works, and then update/rewrite the scripts, but I want to have a stop-gap solution while I'm learning. I don't want to update the router and have everything in a broken state for several days/weeks while I figure out the nftables syntax.

tievolu · August 10, 2022, 10:24am

I just found out that apparently there's no support in 22.03 for firewall script includes - i.e. no way to have a script run at firewall restart/reload to set up custom rules. This fundamentally breaks my setup

dave14305 · August 10, 2022, 11:00am

There does seem to be script includes available, but not very obvious yet. fw4 will load all script includes after the main firewall start as long as the filename isn’t firewall.user and the include stanza has the fw4_compatible 1 option set.

tievolu · August 10, 2022, 11:03am

Well that's certainly good news. Is this documented anywhere yet? The only reference I can find in the release notes just says:

The extra option which allowed to add custom iptables commands does not work any more.

And I found this post:

dave14305 · August 10, 2022, 11:08am

I learned it by following the commits and some github issues.

https://git.openwrt.org/?p=project/firewall4.git;a=commitdiff;h=11256ff0374fb594e31b0a4e3857f3810ba2933d

github.com/openwrt/openwrt

firewall4: design shortcomings in fw4 custom filter chains

opened 11:43AM - 02 Jun 22 UTC

closed 04:14PM - 14 Jul 22 UTC

Shine-

core packages

### Problem description: fw4 custom filter chains ("10-custom-filter-chains.n…ft" or "*.nft") have some logic/design issues. Below are two example cases, and I'm sure there are more. ### Case 1: Background: Firewall rules should always be processed from top to bottom. If one rule is matched, processing should end, so the matched rule can come into effect. This works fine as long as you remain within **only one nft chain**. As soon as you add additional chains, nft behaves differently. A "Drop" rule in any chain will render all "Allow" rules in all previous (lower priority) chains void. This is due to nft's behavior of continuing to process any later chains, even if a rule matches, **until** a Drop is found. So the "top to bottom" approach suddenly applies only to "Drop" rules, not to "Allow" rules anymore. Consequently, this renders fw4's "*.nft" files - which can add additional chains, but not any additional rules in the default chain - quite useless, because they can only be used for "Drop" rules, not for "Allow" rules. ### Problem for Case 1: Due to the nft behavior with multiple chains, at this time, the user doesn't have a possibility to add custom "Allow" rules to fw4. This especially prohibits the user from configuring fw4 like a proper network firewall should be configured: Deny everything unless explicitly allowed - since the "Deny all" last rule will override any previous "Allow" rules from any lower priority chains! ### Case 2: Background: In a stateful firewall, flows that are already handled by a firewall rule should not be sent through the whole rulebase processing again, as this would mean wasting CPU cycles and therefore slowing down firewall performance. This works fine, as long as the default fw4 chains are the **only** chains used. These chains contain an "Accept" rule for "established,related". As soon as the user adds additional chains (without own "established,related" rules), these complete chains are processed for every packet. As already described in Case 1, this re-processing will repeat for each and every chain, as an "Accept" rule will not stop processing later chains. ### Problem for Case 2: By adding simple additional rules as separate nft chains, instead of within the default chains, performance will suffer. ### Suggestion: "*.nft" files aren't currently useful for the purpose one would suppose: quickly adding additional firewall rules that aren't supported in uci/LuCI syntax. While Case 2 is just "inconvenient", but can be solved by adding "established,related" rules to each chain, I don't see how Case 1 can be solved in the current state of fw4 at all. The possibility of adding complete additional nft chains might be useful for special purposes. It is, however, not a feasible way of quickly adding custom firewall rules. Therefore, it should not be the **only** possible way of customizing the fw4 ruleset. Rather, the user should be given an easy possibility of customizing the fw4 default chains, without having to think of all side effects and limits that he might have when using separate nft chains. After all, the standard OpenWRT user, or even the OpenWRT Power User, is not a nft expert!