Nftables ip set swap, how?

papasan · March 17, 2026, 7:34pm

in iptables there was an ‘ipset swap’ command to swap 2 sets, is there anything like this for nftables?

use-case, load up a network list of malicious IPs and CIDR networks to block connections to. when updating the set first need to “flush” to remove the old entries, then “add” new entries. during the time between “flush” and “add” the network is vulnerable. better method would be to load a new set, swap sets, destroy the old set for consistency.

this was do-able with iptables but as far as I can see not with nftables.

efahl · March 17, 2026, 8:39pm

A single nft invocation is atomic, so just group your set flush and adds into a single command. Something like this

$ nft '
   flush set inet fw4 my_set
   add element inet fw4 my_set {
      1.2.3.4, 2.3.4.5, ...
   }
'

Or put your commands in a file and use nft -f file.

papasan · March 17, 2026, 8:57pm

there’s thousands of IPs that are flushed and added to the set when an update happens, sometimes taking a couple of minutes to complete. are you saying that putting “flush” and “add” in a single statement somehow makes it seamless?

edit– it’s been awhile but after buggering with it for a little bit today I recall that dumping it to a text file and running “nft -f” errors out, guessing the command is too long. I have to add them in like this ($MAL is a list, one entry per line)…

        ulimit -s 65536
        nft flush set inet custom mal
        nft add element inet custom mal { $(awk '{print $1 ","}' $MAL) }

…here’s the entire thing in case it’s helpful…

#!/bin/sh
## update local lists and nft sets for firewall usage
set -e #exit on error

DOH='/root/doh.txt'
MAL='/root/mal.txt'
RGX='^([0-9]{1,3}\.){3}[0-9]{1,3}(\/[0-9]{1,2})?' #only IPv4 and CIDR

# make sure we're online before procceeding...
ping -c1 raw.githubusercontent.com >/dev/null

# update DOH list...
if [ -f $DOH ]; then mv $DOH $DOH.old; else touch $DOH.old; fi
wget 'https://raw.githubusercontent.com/crypt0rr/public-doh-servers/refs/heads/main/ipv4.list' -qO - >$DOH
wget 'https://cdn.jsdelivr.net/gh/hagezi/dns-blocklists@latest/ips/doh.txt' -qO - >>$DOH
wget 'https://raw.githubusercontent.com/oneoffdallas/dohservers/master/iplist.txt' -qO - >>$DOH
wget 'https://public-dns.info/nameservers.txt' -qO - >>$DOH
grep -Eo $RGX $DOH |sort -u >$DOH.tmp
mv $DOH.tmp $DOH

# update MAL list...
if [ -f $MAL ]; then mv $MAL $MAL.old; else touch $MAL.old; fi
wget 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level1.netset' -qO - >$MAL
wget 'https://raw.githubusercontent.com/firehol/blocklist-ipsets/master/firehol_level2.netset' -qO - >>$MAL
wget 'https://cdn.jsdelivr.net/gh/hagezi/dns-blocklists@latest/ips/tif.txt' -qO - >>$MAL
wget 'https://raw.githubusercontent.com/stamparm/ipsum/master/levels/3.txt' -qO - >>$MAL
wget 'https://raw.githubusercontent.com/romainmarcoux/malicious-ip/refs/heads/main/full-aa.txt' -qO - >>$MAL
grep -Eo $RGX $MAL |sort -u >$MAL.tmp
mv $MAL.tmp $MAL

# update nft sets if needed...
ulimit -s 65536
if [ "`diff -q $DOH $DOH.old`" ]; then
        nft flush set inet custom doh
        nft add element inet custom doh { $(awk '{print $1 ","}' $DOH) }
fi
if [ "`diff -q $MAL $MAL.old`" ]; then
        nft flush set inet custom mal
        nft add element inet custom mal { $(awk '{print $1 ","}' $MAL) }
fi
#EOF

efahl · March 17, 2026, 9:50pm

A better strategy for things like block lists is to let the set self-flush by giving it a finite lifetime.

table inet test {
        set doh_ipv4 {
                type ipv4_addr
                flags dynamic,timeout
                timeout 7d
                gc-interval 6h
        }
}

Then you can simply add elements with a new expiration time, thus updating their time to live.

$ nft 'add element inet test doh_ipv4 { 1.0.0.1 expires 7d }'

$ nft 'get element inet test doh_ipv4 { 1.0.0.1 }'
table inet test {
        set doh_ipv4 {
                type ipv4_addr
                flags dynamic,timeout
                timeout 7d
                gc-interval 6h
                elements = { 1.0.0.1 expires 6d23h59m6s440ms }

Wait a while, then rerun both commands above and see that the expiration is updated each time you add the element.

With this mechanism you don't have to worry about holes in security during updates, because you never remove anything, just update what's still there or add new ones...

antonk · March 17, 2026, 9:51pm

nftables has known bugs related to large sets which makes atomic sets replacement not practical with large sets. See here the fastest way to load large sets:

github.com/friendly-bits/geoip-shell

lib/geoip-shell-lib-nft.sh

63c66af0b


      
          		}
          
          		# add geoblocking enable rule
          		[ "$no_block" = false ] && printf '%s\n' "add rule inet $geotable $base_geochain jump $geochain comment ${geotag}_enable"
          	done
          
          	:
          }
          
          apply_rules() {
          	print_ipset_rule() {
          		case "$3" in
          			ip) ipset_flags="" ;;
          			net) ipset_flags="flags interval; auto-merge;"
          		esac
          
          		printf %s "add set inet $geotable $1 { type ${4}_addr; $ipset_flags policy $nft_perf; "
          
          		case "$1" in
          			local_*|allow_*)
          				printf %s "elements={ "

papasan · March 18, 2026, 12:09am

sounds like the answer to my question is “no”…

the expiration thing is interesting, I’ll have to give that some thought. Antok, that looks like an amazing script but waaaay more than I want to implement just now.

antonk · March 18, 2026, 12:15pm

When using add element, you are running right into nftables bugs which make it very slow.
I just wanted to point out a proven fast way to import IP lists. Basically:

{
	printf %s "add set inet <table> <set_name> { type <family>_addr; flags <set_flags>; policy <set_policy>; elements={"
	sed '/^$/d;s/$/,/' "<path_to_file>"
	printf '%s\n' " }; }"
} | nft -f -

Replace keywords in <, > brackets with your values. Flags and policy are optional. The sed command prints your newline-separated IP list file as <ip>, <ip>, ... , <ip>.

pavelgl · March 18, 2026, 1:58pm

You could first create a new set, update the firewall rule with the new set name, and then delete the old set.

# Starting point

table inet custom {
        set doh {
                type ipv4_addr
                elements = { 1.2.3.4 }
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                ip daddr @doh drop
        }
}

# Get the name of the old set
old_doh=$(nft list sets | grep doh | awk '{ print $2 }')

# Gather information to obtain the firewall rule handler
handle=$(nft -a list chain inet custom forward | grep \@doh)

# Generate a new set name using the current date, create the set and populate it
new_doh=doh_$(date +%m%d)
nft add set inet custom $new_doh { type ipv4_addr\; }
nft add element inet custom $new_doh { 5.6.7.8 }

# Update the firewall rule
nft replace rule inet custom forward handle "${handle##* }" ip daddr @$new_doh drop

# Delete the old set
nft delete set inet custom $old_doh

# Result

table inet custom {
        set doh_0318 {
                type ipv4_addr
                elements = { 5.6.7.8 }
        }

        chain forward {
                type filter hook forward priority filter; policy accept;
                ip daddr @doh_0318 drop
        }
}

papasan · March 18, 2026, 4:15pm

antonk- understood, yes I realized that one at a time is suuuuuuuuper slow so using ‘awk’ to combine elements. still takes time but not as bad.

I ended up using diff to add/remove instead of flushing the set, hopefully this is stable…

edit updated to avoid blank changes…

ulimit -s 65536
diff $DOH $DOH.old >$DOH.dif
if grep -q '^> ' $DOH.dif; then nft delete element inet custom doh { $(grep '^> ' $DOH.dif |awk '{print $2 ","}') }; fi
if grep -q '^< ' $DOH.dif; then nft add element inet custom doh { $(grep '^< ' $DOH.dif |awk '{print $2 ","}') }; fi
diff $MAL $MAL.old >$MAL.dif
if grep -q '^> ' $MAL.dif; then nft delete element inet custom mal { $(grep '^> ' $MAL.dif |awk '{print $2 ","}') }; fi
if grep -q '^< ' $MAL.dif; then nft add element inet custom mal { $(grep '^< ' $MAL.dif |awk '{print $2 ","}') }; fi

antonk · March 18, 2026, 4:30pm

My recommendation: if possible, avoid elements manipulation entirely. Rather:

Remove rules which reference the old set and the set itself (can be done atomically)
Create a new set and rules for it (can be done atomically)

Yes, this causes your rules to not be active for the time of the update. However that time should be much shorter than with your current implementation.

P.s. if the IPs in your $DOH and $DOH_OLD files are unsorted, there might be cases where the actual lists are the same but the diff command will show them as different. If you want to account for that, use something like this:

github.com/friendly-bits/geoip-shell

lib/geoip-shell-lib-common.sh

a63544e50


      
          		END{
          			if(r==1){exit 1}
          			if(!a&&!b){exit 0}
          			if(!a||!b){exit 1}
          			for (a in A) if (!(a in B)){exit 1}
          			exit 0
          		}
          	' "$1" "$2"
          }
          
          # compares lines in files $1 and $2, regardless of order
          # discards empty lines
          # returns 0 for no diff, 1 for diff, 2 for error
          compare_files() {
          	[ -f "$1" ] && [ -f "$2" ] || { echolog -err "compare_files: file '$1' or '$2' does not exist."; return 2; }
          	awk_cmp "$1" "$2"
          }
          
          # compares lines in file $1 and string $2, regardless of order
          # discards empty lines
          # returns 0 for no diff, 1 for diff, 2 for error

(You will also need the awk_cmp() function from the same script. replace the call to echolog with your own code, e.g. echo "Error: compare_files: file '$1' or '$2' does not exist." >&2).

But then again, if possible, avoid all that and just completely replace the set as explained above.