Goal is to have a firwall rule that allows me to block or allow traffic von domains like "https://www.reuters.com" . Domains like that have not a single IP, but somtime use dozens to deliver content / a html page to the browser.
In the past, dnsmasq-full and fw3 and ipsets had solution for this.
Defining and initalizing an ipset in fw3 , and dnsmasq populated the ipset list with IPs.
fw3 rules could work with this ipsets. All good.
No good (working for me / my issue) is current openWRT Release 23.03 which uses fw4 and dnsmasq 2.86.
fw4 is based on nftables and seems to have lost the abilty to work with iplist.
dnsmasq 2.86 does not work with nftable.
I tried all combination, followed all available documentation, threads etc, but I am not able to reach my goal in openWRT 22.03.03.
This are the links to my major info sources - to make research easier:
Thanks and Kudos to everyone how shares his knowledge how to solve this or confirms that this goal is not reachable within 22.03.03 commited software packages.
Apparently nftables support requires dnsmasq 2.87+, which is likely not going to happen until the next major OpenWrt release, until then you need to resolve the domains and populate the sets using a script as explained in the above links, or you can try to install an OpenWrt snapshot build.
thx 4 reply.
Could you tell me which Link explains it right?
You say (interpretation) .. use / initialize nftables in fw4 , ipsets in dnsmasq and let them work together with the help of a script?
If yes .. my problem starts that I thought I need to initalize ipset with the help of the firewall, and then, after the ipset exists, dnsmasq will use the ipsets. or in other words .. i fail to initalize ipset (maybe thats my problem) because fw4 is not doing that for me anymore.
fw4 is based on nftables providing own built-in IP set implementation which is incompatible with the old IP sets.
Although configs and some commands may keep the word "ipset" in their name, they no longer use the old IP set back-end.
The problem is that dnsmasq 2.86 does not support the new IP set implementation.
thank you again. i see that it is currently needed to get deep inside the system / technology to make this work. for the moment i think this goes unforunatly to far beyond my know how / capabilities. i could try to type in the script .. but i do not understand whats going on. it seems that ipsets where easier to understand / handle for endusers ..
Maybe another usefull workaround I found that might work for small websites / domain you want to filter.
A user @ superuser asked how to find out the IP Adresses of wikipedia and got a nice answer.
If you define a firewall rule using this two ip (ranges) .. you can accept / reject requests for e.g. www.wikipedia.org
Did you check whether the nftset option satisfies your needs? If so, you can simply build your own image from Master, which is already at dnsmasq 2.88, or build from 23.03.3 stable release while editing the dnsmasq files to use version 2.87 or 2.88. I did it both ways to route the traffic to some streaming services around VPN.
If you need help with defining the correct rules, I am sure u/ pavelgl can help you out. He has provided me and a lot of others with the correct answers or the necessary hints and groundwork around fw4 and nftables.
you are right: current version of dnsmasq (2.86) and fw4 in stable owrt is not working together, cannot use the old ipset functionality.
but.
fw4 which is based on nftables does support nft set and you can use simple tools like resolveip to resolve a domain and add ips to nft set. then you can set fw4 rules based on nft set similar to ipset + dnsmasq + fw3.
it certainly will not be as comfortable as using native dnsmasq + nftset because you somehow need to run resolveip, and depending on the domains it can be one time act or if the ips are changing (e.g. in case of cloud service, CDN behind the domain) you need to set some kind of scheduled dns resolution (i.e. every X hours/mins).
or you can use dnsmasq-full with ipset, and copy ips from ipset into nft set.
thx 4 this infos! Building my own image is beyond my capabilities. I already tried to inject 2.88 into 23.03.3 but bloody failed. My Linux / openWRT know how is much to low to understand what i am doing and why it fails
The Link contains valuable info .. need to have a deeper look on that
Hi, thank you for the detailed explanations. This confirms that the goal can be achived, i suspected that, but I also suspected that its not an easy task / out ob the box. It seems to be a hands on job for a real Linux / openWRT insider, who is confident with self made (schedulded) scripts.
I am able to install resolveip, and to editi config files (sometimes the wrong ones) of firewall and dnsmasq. Edit config files .. means putting something in I find ready made somewhere else
So yes, there are solutions, always Creating ones opwnWRT Version surly works. Integrating new (beta) software packages surly works.
But the questions was - can one do it with the software packages that come with openWRT 22.03.03, staying within tested production sw versions that are compatible with each other. Can one follow a documented best pratce way to reach the goal? I think the goal is not way off from what people think openWRT can deliver close "out of the box" / standard feature.
Its a ittle bit frustrating that this feature already existed and will be available again .. but not now in 22.03.03.
Proud owner of the r7800 nighthawk. Bought it after reviewing the forums. Great device.
I started 2 weeks ago getring deeper into openWRT because a customer wants to protect important data. Therefore i created a safe vlan / network with pcs / internet browsers (most important software is a cloud app) to get outside, but i want to limit the browsers capabilities, which is not as easy as i thought. Allow one domain and eliminating all others .. never thought that would take me a week to find out that you currently cant do that with basic openWRT installation and basic modification skills (no linux experience, no sw development, no scripts)
I think that this thread already delivered the pieces of the solution, but i am not able to put them together to a working solution.
If I undertand right:
I need to install resolveip IP package.
-> can do
need to create / initalize an suitable nfttable to reach the goal somehow somewhere.
-> not sure how and where. any help / hint / link apprecaited
need resolveip to fill the nftable with IP4 adresses .. lets stick to the example, of www.reuters.com
-> no idea how to do that
need a script (?) that let resolveip do this onece a hour or so to keep the table updated
-> no idea how to do that, close connection to 3). I never did a script / cronejob, but seems solveable with some generic help.
need a firwall rule that depends on that filled nftable to decide accept / reject
-> seems solveable with a little research. can not be implemented using LuCI i think, handwork.
As this might be of interest for other people as well using openWRT 22.03.03 .. if you / the community could help me with specific info regarding point 2 - 5 (or telling me the right points) I would implement and compile the info and document properly how to reach the goal in 22.03.03. Goal should be to make it as easy as possible for other users to do the same. step by step starting with an unmodified 22.03.03 image
any chance? thx to all replys untill now that brought me here / to this idea to proceed.
Yes, would love to use your proposed solution!
Following questions:
In the proposed solution Ip Sets are used. I thought this does not work with nftables. Taking a close look I asume the "ipset" package does not need to be installed, is not used. IP Set just names the concept that is used, not the package. So IP Sets concept is used in proposed solution. So the instructions in openwrt.org/docs/guide-user/firewall/fw3_configurations/dns_ipset are principally suitable for my problem !. These instructions were written for fw3 and need to e modified to work for fw4 (I think). Anyway .. please confirm.
In proposed solution you explain that fw4 has an built-in IP set implementation. Check. "initalize / invoke" this built-in IP set in fw4 is done in the fw3 code under "# Configure IP sets" and must be used in the proposed solution? it looks like the code is typed in once in the terminal. tried it, seems to work. also the domain (example: "reuters.com" ) is defined here. how does the result stay on the router (after reboot) ? is it "saved" somewhere? can it be edited again? where? how?
If the zwo questions / points are clear, the nftable prepared to host the IP Sets should exist in the memory / Kernel ( I asume) . So next step is to fill the table with the help of resolveip .. I think. I see a section in the fw3 link that says "# Populate IP sets", one command follows. It uses ipset ... that is not installed. I asume we are leaving the fw3 instructions here, we use resolveip to populate. for me is unclear .. how.
I tried to understand what part is done in your code. it comes in your post after me saying " i failed to initalize ipset" . First - I do not understand how to get your code into system. the final line sh dns-ipset.sh
seems to a command in the (putty) shell. but the rest .. if i put the command / variable definition URL="https://openwrt.org/_export/code/docs/guide-user" in putty followed by cat << EOF > dns-ipset.sh .. that leads nowhere. You see .. no experience with Linux and scripts, shells etc. leads to no understanding what to do with your code ... sorry
Similar problem with the code shown in the fw3 link. the code in the "# Filter LAN client traffic with IP sets" section does not work in puttys terminal window. it looks like a script to me, some programing instructions. might be a script sorry again, I do not know how to handle that. Every hint welcome.
I hope you see that I really would like to try out your proposed solution.
Currently I do not completly understand where / how the nftable is defined / invoked / can be edited. I can not use the code to generate the firewall rule listed in the fw3 document. I can not use your code. I do not understand which parts of the fw3 link / document I can / should use and which parts are not applicable.
I gladly read documentation / do self study, but I think you see that I need some more hints / questions answerded. THX in advance!!
# approach #1:
# edit /etc/config/firewall
# create ipset config (yes it may sound a bit confusing
# but it is indeed called ipset and not nftset. but bear with me on 22.x it will
# create an nft set in the background)
config ipset
option name 'myipset'
option family 'ipv4'
option match 'dest_ip' # <- will match against destination ip, useful to block based on target ip
# option loadfile '/tmp/doh4.txt' # <- optional
# list entry <ip address of domain1> # <- optional
# list entry <ip address of domain2> # <- optional
config rule
option name 'Block Domains I dislike'
option ipset 'myipset dest'
option family 'ipv4'
option dest 'wan'
option dest_port '443' # <- optional
option target 'DROP'
option src '*'
what it does:
1st config creates a nft set called myipset. either you create a file with IPs, or list entries in the config. pick only one.
this will block all access targeting IPs in the ipset, optionally only if target port is 443.
# approach #2
# create the ipset config without loadfile and entry. so IPs will be added separately.
# add the same rule
$ opkg update && opkg install resolveip
$ cat << EOF >> /root/resolve2nft.sh
#!/bin/sh
resolveip -4 www.reuters.com | while read -r ip; do
nft add element inet fw4 myipset $ip
done
EOF
$ chmod + /root/resolve2nft.sh
$ cat << EOF >> /etc/crontabs/root
*/5 * * * * /root/resolve2nft.sh
EOF
what it does:
with firewall config you create an empty ipset, and a rule which will again block traffic targeting IPs.
then every 5 mins crond will run resolve2nft.sh which will resolve the current IPs associated with www.reuters.com and insert into myipset.
NOTE: these are high level, poorly tested (if at all) ideas.
does not mean they are optimal, surely there are rough edges and pretty sure made typos.
Yes, it is a shell function in this specific case.
The concept is the same as IP set is still a set of IPs, albeit implemented differently.
Already done.
The UCI syntax defining IP sets is basically the same for both fw3 and fw4.
The IP sets are populated at startup with Hotplug extras when the system becomes online and then updated on a schedule.
OpenWrt provides flash wear leveling by using tmpfs for its data directory.
If your setup is unaffected by wearing, you can change the paths /var/ipset-* to a persistent location, e.g. /etc/nftables.d/ipset-*, so the IP sets can survive reboot and do not require to be populated at startup.
That code uses the here-document shell feature to create a script with the content combined from the code blocks on the respective wiki pages.
It is not going to run until you close the EOF delimeter.
You are expected to copy-paste the entire block of code to the terminal as is.
If necessary, you can also customize the final script before running it.
It is not going to work until you close the command list with the done word.
Multi-line copy-paste should work in popular terminal emulators.
I will need some time to get through, (and find the place where resolveip does its job, i am curious). Will give it a try definitly. And document the whole for beginners if i succed. the solution also includes lots of generic learnings, very welcome, also thx for that.
