How to mark IP addresses not belong to a IPset in 22.03

utopinux · January 26, 2023, 7:38am

Hi
This used to work in 21.03 but not anymore in 22.03 firewall says 'storage' and 'extra' are not supported in fw4. what should I change now to make it work again ( except going back to 21.03))

config rule
        option name 'Mark Not IPv4 VN'
        option family 'ipv4'
        option src 'lan'
        option target 'MARK'
        option set_mark '1'
        option extra ' -m set ! --match-set ipv4vn dst'
        list proto 'all'
        option dest '*'
        option enabled '0'

config  ipset
        option  name            'IPv4VN'
        option  match           'src_net'
        option   family		  'ipv4'
        option  storage         'hash'
        option  enabled         '1'
        list    entry           '103.252.0.0/22'
        list    entry           '115.146.120.0/21'
        list    entry           '115.165.160.0/21'

pavelgl · January 26, 2023, 9:17am

config ipset
        option name 'ipv4vn'
        option match 'dest_net'
        list entry '103.252.0.0/22'
        list entry '115.146.120.0/21'
        list entry '115.165.160.0/21'

config rule
        option name 'Mark Not IPv4 VN'
        option family 'ipv4'
        option src 'lan'
        option target 'MARK'
        option set_mark '1'
        option ipset '!ipv4vn'
        list proto 'all'
        option dest '*'

To check the results:

nft list set inet fw4 ipv4vn
nft list chain inet fw4 mangle_prerouting

utopinux · January 26, 2023, 2:30pm

Thank you for you help! However I still dont have the results: Addresses in IPset shall go through the Wan and the rest shall go through the VPN called nordlynx. Please see the config below in /etc/config/network

config route
	option interface 'nordlynx'
	option target '0.0.0.0/0'
	option table '201'

config rule
	option mark '1'
	option lookup 'vpn.nordlynx'

pavelgl · January 26, 2023, 3:31pm

Please, change this to

config rule
        option mark '0x1'
        option lookup '201'

and restart the network service.

Then post the output of:

nft list set inet fw4 ipv4vn; nft list chain inet fw4 mangle_prerouting; \
ip rule list; ip route list table 201

utopinux · January 26, 2023, 3:42pm

nft list set inet fw4 ipv4vn


table inet fw4 {
        set ipv4vn {
                type ipv4_addr
                flags interval
                auto-merge
                elements = { 1.52.0.0/14, 14.160.0.0/11,
                             14.224.0.0/11, 27.0.12.0/22,
                             27.0.240.0/22, 27.2.0.0/15,
                             27.64.0.0/12, 27.118.16.0/20,
                             42.112.0.0/13, 43.239.148.0/22,
                             43.239.188.0/22, 43.239.220.0-43.239.227.255,
                             45.117.76.0-45.117.83.255, 45.117.160.0-45.117.179.255,
                             45.118.136.0-45.118.151.255, 45.119.76.0-45.119.83.255,
                             45.119.108.0/22, 45.119.212.0-45.119.219.255,
                             45.119.240.0/22, 45.120.228.0/22,
                             45.121.24.0/22, 45.121.152.0/22,
                             45.121.160.0/22, 45.122.220.0/22,
                             45.122.232.0-45.122.255.255, 45.123.96.0/22,
                             45.124.84.0-45.124.95.255, 45.125.200.0-45.125.211.255,
                             45.125.236.0/22, 45.126.92.0-45.126.99.255,
                             45.127.252.0/22, 45.251.112.0/22,
                             45.252.240.0-45.252.251.255, 45.254.32.0/22,
                             47.117.156.0/22, 49.156.52.0/22,
                             49.213.64.0/18, 49.246.128.0-49.246.223.255,
                             58.84.0.0/22, 58.186.0.0/15,
                             59.153.216.0-59.153.255.255, 61.14.232.0/21,
                             61.28.224.0/19, 101.53.0.0/18,
                             101.96.12.0/22, 101.96.64.0/18,
                             101.99.0.0/18, 103.1.200.0/22,
                             103.1.208.0/22, 103.1.236.0/22,
                             103.2.220.0/22, 103.3.244.0-103.3.255.255,
                             103.4.128.0/22, 103.5.30.0/23,
                             103.5.204.0-103.5.211.255, 103.7.36.0-103.7.43.255,
                             103.7.172.0/24, 103.7.177.0/24,
                             103.7.196.0/24, 103.8.13.0/24,
                             103.9.0.0/21, 103.9.76.0-103.9.87.255,
                             103.9.156.0/22, 103.9.196.0-103.9.207.255,
                             103.9.212.0/22, 103.10.44.0/22,
                             103.10.88.0/22, 103.10.212.0/22,
                             103.11.172.0/22, 103.12.104.0/22,
                             103.13.76.0/22, 103.15.48.0/22,
                             103.16.0.0/22, 103.17.88.0/22,
                             103.17.236.0/22, 103.18.4.0/22,
                             103.18.176.0/22, 103.19.164.0/22,
                             103.20.144.0/21, 103.21.120.0/22,
                             103.21.148.0/22, 103.23.144.0/22,
                             103.24.244.0/22, 103.26.252.0/22,
                             103.27.60.0-103.27.67.255, 103.27.236.0/22,
                             103.28.36.0/22, 103.28.136.0/22,
                             103.28.172.0/22, 103.30.36.0/22,
                             103.31.120.0/21, 103.35.64.0/22,
                             103.37.28.0-103.37.35.255, 103.38.136.0/22,
                             103.39.92.0-103.39.99.255, 103.42.56.0/22,
                             103.45.228.0/22, 103.45.236.0/22,
                             103.47.192.0/22, 103.48.76.0-103.48.83.255,
                             103.48.188.0-103.48.195.255, 103.52.92.0/22,
                             103.53.88.0/22, 103.53.168.0/22,
                             103.53.228.0/22, 103.53.252.0/22,
                             103.54.248.0/22, 103.56.156.0-103.56.171.255,
                             103.57.104.0/22, 103.57.112.0/22,
                             103.57.208.0/22, 103.57.220.0/22,
                             103.60.16.0/22, 103.61.48.0/22,
                             103.62.8.0/22, 103.63.104.0-103.63.123.255,
                             103.63.212.0/22, 103.66.152.0/22,
                             103.68.68.0-103.68.83.255, 103.68.240.0/20,
                             103.69.192.0/22, 103.70.28.0/22,
                             103.74.100.0/22, 103.74.112.0-103.74.123.255,
                             103.75.176.0-103.75.187.255, 103.77.160.0-103.77.171.255,
                             103.78.84.0/22, 103.79.140.0-103.79.147.255,
                             103.81.80.0/21, 103.82.20.0-103.82.39.255,
                             103.82.196.0/22, 103.84.76.0/22,
                             103.88.108.0-103.88.123.255, 103.89.84.0-103.89.91.255,
                             103.89.120.0/22, 103.90.220.0-103.90.235.255,
                             103.92.16.0/22, 103.92.24.0-103.92.35.255,
                             103.94.176.0/22, 103.95.168.0/22,
                             103.95.196.0/22, 103.97.124.0/22,
                             103.97.132.0/22, 103.98.148.0-103.98.155.255,
                             103.98.160.0/22, 103.99.0.0/22,
                             103.99.228.0/22, 103.99.244.0/22,
                             103.99.252.0/22, 103.100.160.0/22,
                             103.100.228.0/22, 103.101.32.0/22,
                             103.101.76.0/22, 103.101.160.0/22,
                             103.102.20.0/22, 103.103.116.0/22,
                             103.104.24.0/22, 103.104.116.0-103.104.123.255,
                             103.106.220.0-103.106.227.255, 103.107.180.0/22,
                             103.107.200.0/22, 103.108.100.0/22,
                             103.108.132.0-103.108.139.255, 103.109.28.0-103.109.43.255,
                             103.110.84.0/22, 103.111.236.0/22,
                             103.111.244.0/22, 103.112.124.0/22,
                             103.113.80.0/22, 103.113.88.0/22,
                             103.114.104.0/22, 103.115.166.0/24,
                             103.116.100.0-103.116.107.255, 103.117.240.0/21,
                             103.119.84.0/22, 103.121.88.0/22,
                             103.124.56.0/22, 103.129.80.0-103.129.91.255,
                             103.129.188.0/22, 103.130.220.0/22,
                             103.131.72.0/22, 103.140.40.0/23,
                             103.141.176.0/23, 103.192.236.0/22,
                             103.195.236.0-103.195.243.255, 103.196.16.0/22,
                             103.196.236.0/22, 103.196.244.0-103.196.251.255,
                             103.199.4.0-103.199.79.255, 103.200.20.0/22,
                             103.200.60.0/22, 103.200.120.0/22,
                             103.205.96.0-103.205.107.255, 103.206.212.0-103.206.219.255,
                             103.207.32.0/21, 103.211.212.0/22,
                             103.213.120.0/22, 103.214.8.0/22,
                             103.216.72.0/22, 103.216.112.0/22,
                             103.216.120.0-103.216.131.255, 103.219.180.0/22,
                             103.220.68.0/22, 103.220.84.0/22,
                             103.221.86.0/24, 103.221.212.0/22,
                             103.221.220.0-103.221.231.255, 103.223.4.0/22,
                             103.225.236.0/22, 103.226.108.0/22,
                             103.226.248.0/22, 103.227.112.0/22,
                             103.227.216.0/22, 103.228.20.0/22,
                             103.229.192.0/22, 103.231.148.0/22,
                             103.232.52.0-103.232.63.255, 103.232.120.0/22,
                             103.233.48.0/22, 103.234.36.0/22,
                             103.235.208.0/21, 103.237.60.0-103.237.67.255,
                             103.237.96.0/22, 103.237.144.0/21,
                             103.238.68.0-103.238.83.255, 103.238.208.0/21,
                             103.239.32.0/22, 103.239.116.0-103.239.123.255,
                             103.241.248.0/22, 103.242.52.0/22,
                             103.243.104.0/22, 103.243.216.0/22,
                             103.245.148.0/22, 103.245.248.0/21,
                             103.246.104.0/24, 103.248.160.0/21,
                             103.249.100.0/22, 103.250.24.0/22,
                             103.252.0.0/22, 103.252.252.0/22,
                             103.253.88.0/22, 103.254.12.0-103.254.19.255,
                             103.254.40.0/22, 103.254.216.0/22,
                             103.255.84.0/22, 103.255.236.0/22,
                             110.35.64.0/20, 110.44.184.0/21,
                             111.65.240.0/20, 111.91.232.0/22,
                             112.72.64.0/18, 112.78.0.0/20,
                             112.109.88.0/21, 112.137.128.0/20,
                             112.197.0.0/16, 112.213.80.0/20,
                             113.20.96.0/19, 113.22.0.0-113.23.127.255,
                             113.52.32.0/19, 113.61.108.0/22,
                             113.160.0.0/11, 115.72.0.0/13,
                             115.84.176.0/21, 115.146.120.0/21,
                             115.165.160.0/21, 116.68.128.0/21,
                             116.96.0.0/12, 116.118.0.0/17,
                             116.193.64.0/20, 116.212.32.0/19,
                             117.0.0.0/13, 117.103.192.0/18,
                             118.68.0.0/14, 118.102.0.0/21,
                             118.107.64.0/18, 119.15.160.0/19,
                             119.17.192.0/18, 119.18.128.0/20,
                             119.18.184.0/21, 119.82.128.0/20,
                             120.50.184.0/21, 120.72.80.0/21,
                             120.72.96.0/19, 120.138.64.0/20,
                             121.50.172.0/22, 122.102.112.0/22,
                             122.201.8.0/21, 123.16.0.0/12,
                             124.158.0.0/20, 125.212.128.0/17,
                             125.214.0.0/18, 125.234.0.0/15,
                             125.253.112.0/20, 137.59.24.0-137.59.47.255,
                             137.59.104.0/22, 137.59.116.0/22,
                             144.48.20.0/22, 146.196.64.0/22,
                             150.95.16.0/22, 150.95.104.0-150.95.127.255,
                             157.119.244.0-157.119.251.255, 163.44.192.0/22,
                             163.44.200.0/24, 163.44.204.0/22,
                             171.224.0.0/11, 175.106.0.0/22,
                             180.93.0.0/16, 180.148.0.0/21,
                             180.148.128.0/20, 182.161.80.0/20,
                             182.236.112.0/22, 182.237.20.0/22,
                             183.80.0.0-183.81.127.255, 183.90.160.0/21,
                             183.91.0.0/19, 183.91.160.0/19,
                             202.0.79.0/24, 202.4.168.0/24,
                             202.4.176.0/24, 202.6.2.0/24,
                             202.6.96.0/23, 202.9.79.0-202.9.80.255,
                             202.9.84.0/24, 202.37.86.0/23,
                             202.43.108.0/22, 202.44.137.0/24,
                             202.52.39.0/24, 202.55.132.0/22,
                             202.58.245.0/24, 202.59.238.0/23,
                             202.59.252.0/23, 202.60.104.0/21,
                             202.74.56.0/24, 202.74.58.0/23,
                             202.78.224.0/21, 202.87.212.0/22,
                             202.92.4.0/22, 202.94.88.0/23,
                             202.124.204.0/24, 202.130.36.0/23,
                             202.134.16.0/21, 202.134.54.0/24,
                             202.143.108.0/22, 202.151.160.0/20,
                             202.160.124.0/23, 202.172.4.0/23,
                             202.191.56.0/22, 203.8.127.0/24,
                             203.34.144.0/24, 203.77.178.0/24,
                             203.79.28.0/24, 203.99.248.0/22,
                             203.113.128.0/18, 203.128.240.0/21,
                             203.160.0.0/23, 203.160.132.0/22,
                             203.161.178.0/24, 203.162.0.0/16,
                             203.167.12.0/22, 203.170.26.0/23,
                             203.171.16.0/20, 203.176.160.0/21,
                             203.189.28.0/22, 203.190.160.0/20,
                             203.191.8.0/21, 203.191.48.0/21,
                             203.196.24.0/22, 203.201.56.0/22,
                             203.205.0.0/18, 203.210.128.0/17,
                             210.2.64.0/18, 210.86.224.0/20,
                             210.211.96.0/19, 210.245.0.0/17,
                             220.231.64.0/18, 221.121.0.0/18,
                             221.132.0.0/18, 221.133.0.0/19,
                             222.252.0.0/14, 223.27.104.0/21 }
        }
}

nft list chain inet fw4 mangle_prerouting

table inet fw4 {
        chain mangle_prerouting {
                type filter hook prerouting priority mangle; policy accept;
                iifname "br-lan" ip daddr != @ipv4vn counter packets 7489 bytes 1433915 meta mark set 0x00000001 comment "!fw4: Mark Not IPv4 VN"
                iifname "WG" ip daddr != @ipv4vn counter packets 0 bytes 0 meta mark set 0x00000001 comment "!fw4: Mark Not IPv4 VNwg"
        }
}

ip rule list

0:      from all lookup local
1:      from all fwmark 0x1 lookup vpn.nordlynx
32766:  from all lookup main
32767:  from all lookup default

ip route list table 201

default dev nordlynx proto static scope link metric 30

pavelgl · January 26, 2023, 4:00pm

Well, that seems right to me.

All connections from LAN (br-lan) to destination IPs not listed in the set are obviously marked and should be routed through the nordlynx interface.

What is the problem? Does everything go through the vpn or through wan?

utopinux · January 26, 2023, 5:02pm

Indeed it works. The site i used to test was not in the list. Thanks alot.

system · February 5, 2023, 5:03pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.