Mwan3 rules with ipset

Since they resolve the same IP, it doesn't matter.

Something interesting is here:

-A mwan3_iface_wwan -i wlan0 -m mark --mark 0x0/0xff00 -m comment --comment wwan -j MARK --set-xmark 0x200/0xff00

It marks packets coming from interface wlan0 and routes them into table 2 (wan2).

Sorry, now I don't understand, what means table 2. So what is 'wan2' from mwan3? What is interface wlan0 and why have you pinged via it?

Ok I try to explain what is my configuration.
I have eth0.2 as wan1 in mwan3 and it is a normal ADSL line. I also have wlan0 as wan2 in mwan3 and it is a WiFi LTE router but the SIM is a company SIM so it is behind proxy.
What I need is just when I request some specific URL I want to use just eth0.2 aka wan1.

OK, got it.

It is strange behavior, because tracert from PC doesn't show next gateway. Could you check tracert for another site?

Ok, I will try when I'm at home, now I'm at work and I can't access to router.
In the meantime do you think this rule

config rule 'myrule'
    option sticky ‘'
    option timeout ‘'
    option ipset 'myrule'
    option dest_port ''
    option proto ''
    option use_policy 'wan1_only'

is correct? Or I must specify also sticky to get ipset work?

Sticky option needs to 'remember' previous route and to keep it. You don't need it. So with option equaling zero there is no limitation of selecting route. Please, make single uniform quotes.

If it use eth0.2 (wan1) it is show the next gateway but if it use wlan0 (wan2) the next gateway is not showed....
is it strange?

traceroute google.com
traceroute to google.com (216.58.206.78), 64 hops max
  1   192.168.2.1  0,401ms  0,281ms  0,279ms 
  2   10.161.xx.xx  94,229ms  189,777ms  190,060ms 
  3   10.204.xx.xx  61,579ms  58,405ms  69,741ms 
  4   172.22.5.130  50,203ms  59,761ms  50,225ms 
  5   172.22.20.49  59,480ms  50,041ms  69,844ms 
  6   172.31.49.17  70,021ms  69,937ms  59,895ms 
  7   172.31.45.125  79,944ms  79,771ms  70,308ms 
  8   *  *  * 
  9   172.22.3.105  61,490ms  69,945ms  80,586ms 

traceroute google.com
traceroute to google.com (216.58.205.78), 64 hops max
  1   192.168.2.1  0,375ms  0,391ms  0,335ms 
  2   192.168.1.1  1,099ms  0,791ms  0,677ms 
  3   *  *  * 
  4   172.17.144.82  16,613ms  10,350ms  9,951ms 
  5   172.17.145.48  10,878ms  11,679ms  12,673ms 
  6   172.31.49.17  76,555ms  79,693ms  79,965ms 
  7   93.186.128.213  26,872ms  24,509ms  25,431ms 
  8   172.31.43.14  72,768ms  79,856ms  79,694ms

but the rules with ipset not works, I run two times traceroute of URL in ipset and this is the result:

traceroute xxxxxx.com
traceroute to xxxxx.com (85.114.xx.xx), 64 hops max
  1   192.168.2.1  0,430ms  0,316ms  0,309ms 
  2   10.161.xx.xx  60,716ms  249,954ms  49,947ms 
  3   10.204.xx.xx  89,940ms  49,961ms  49,961ms 
  4   172.17.144.156  10,702ms  9,663ms  12,022ms 
  5   172.17.145.126  13,657ms  11,430ms  12,451ms 
  6   172.31.49.17  79,684ms  70,979ms  108,959ms 

traceroute xxxx.com
traceroute to xxxx.com (85.114.xx.xx), 64 hops max
  1   192.168.2.1  0,569ms  0,412ms  0,414ms 
  2   192.168.1.1  1,137ms  0,840ms  0,725ms 
  3   *  *  * 
  4   172.22.5.130  59,527ms  59,758ms  59,841ms 
  5   172.17.145.122  14,964ms  12,058ms  11,953ms 
  6   172.31.49.17  70,689ms  70,062ms  69,921ms 
  7   172.31.45.125  110,097ms  139,451ms  121,635ms 
  8   172.31.43.14  78,312ms  120,268ms  79,514ms

sometimes goes on eth0.2(wan1) and sometimes on wlan0(wan2)

OK, is it from router itself?

Check also: https://stackoverflow.com/questions/48592840/applying-ipset-in-lede

  1. Check, whether dnsmasq-full is installed:
    opkg list-installed | grep dnsmasq
  2. Install, if not.
  3. Check ipset by running command:
ipset -L myipset

No it is from pc.

opkg list-installed | grep dnsmasq return

dnsmasq - 2.73-1

so must I unistall it and install dnsmasq-full?

ipset -L myipset return:

Name: myipset
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 timeout 3600
Size in memory: 8248
References: 1
Members:

OK, so you see your ipset correctly, members present?

Check also connection with your sites, and post only output of rule, marking packets with specific mark, check counters:
iptables -nvL -t mangle

Give also output of:
opkg list-installed | grep ipset

No members is empty...

ipset - 6.24-1
kmod-ipt-ipset - 3.18.23-1

Sorry I not understand what I have to do...

I reboot the router and then DHCP Server not works.
I deleted the ipset entry in /etc/dnsmasq.conf rebooted again the router and now DHCP Server works,
so there is something in my dnsmasq strange...

OK, install dnsmasq-full. Check also quotes, as I've already mentioned.

Output of command iptables -nvL -t mangle shows not only rules, but counters of packets. Try to access your sites in list of ipset, counters should not be equal to zero. So run this command AFTER accessing your sites.

Try also manually to create ipset: add only one IP.

ipset create myipset hash:ip
ipset add myipset X.Y.Z.T
ipset test X.Y.Z.T

https://linux.die.net/man/8/ipset

So the logic is following:

  1. try to create ipset manually, and check, whether routing works.
  2. verify creation of ipset by dnsmasq.

Ok I will try to create ipset manually and then test it.
But can I put an URL instead of an IP when I use ipset add ?

Please can you explain what do you mean with check quotes?

Try with IP initially. After that add lines to config of dnsmasq, try to get site (send query to dnsmasq for resolving), and check, whether ipset is updated.

In your config they were different quotes, see above:

I can't create ipset manually

ipset create myipset hash:ip
ipset v6.24: Set cannot be created: set with the same name already exists
root@OpenWrt:~# ipset list myipset
Name: myipset
Type: hash:ip
Revision: 4
Header: family inet hashsize 1024 maxelem 65536 timeout 3600
Size in memory: 8248
References: 1
Members:
root@OpenWrt:~# ipset destroy myipset
ipset v6.24: Set cannot be destroyed: it is in use by a kernel component

I change the name of ipset, this is the result:

ipset create mineipset hash:ip
root@OpenWrt:~# ipset add mineipset 85.114.134.142
root@OpenWrt:~# ipset test mineipset 85.114.134.142
85.114.134.142 is in set mineipset.

I access the sites in ipset and iptables -nvL -t mangle shows:

Chain mwan3_policy_wan1only (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   40  1714 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            mark match 0x0/0xff00 /* wan 1 1 */ MARK xset 0x100/0xff00

and pkts increase everytime I access the sites in ipset

So I think it works

But now I have to solve the problem to do it in dnsmasq.conf

@ulmwind are you there?
Please can you help me, I do not know what to do....

Thank you so much

Someone can help me?

OK, so now what is your question? Please, try to access sites from your list, and see, whether dnsmasq fills ipset with IP.

same problem

Hi All,

Thanks, @ulmwind & @gfucka for your inputs, I was able to configure my router to do ipset based conditional routing to 2 WAN connections. I wrote an article1 on this topic. Hope someone will find that useful :slight_smile:

3 Likes