Two WANs. Reply on request to the input interface

Installing and Using OpenWrt Network and Wireless Configuration
1

Hello!
I added two routing tables

echo "10 wan_a" >> /etc/iproute2/rt_tables
echo "20 wan_b" >> /etc/iproute2/rt_tables

I added two rules for sent mark trafic to routing tables

config rule
	option name 'mark0x2'
	option mark '0x2'
	option priority '100'
	option lookup 'wan_a'

config rule
	option name 'mark0x3'
	option mark '0x3'
	option priority '100'
	option lookup 'wan_b'

I added two routes

config route 'wan_a'
	option name 'wan_a'
	option interface 'wan_a'
	option table 'wan_a'
	option target '0.0.0.0/0'

config route 'wan_b'
	option name 'wan_b'
	option interface 'wan_b'
	option table 'wan_b'
	option target '0.0.0.0/0'

I checked thet route correctly chosen for mark.

root@OpenWrt:~# ip route get 8.8.8.8 mark 2
8.8.8.8 via <GW_1> dev wan_a table wan_a src <WAN_A_IP> mark 2 uid 0 
    cache
root@OpenWrt:~# ip route get 8.8.8.8 mark 3
8.8.8.8 via <GW_2> dev wan_b table wan_b src <WAN_B_IP> mark 3 uid 0 
    cache

Then i need sent reply to same input interface.
As i understand i need

  1. mark input packet
  2. save mark to conn
  3. restore mark from conn on reply packet

So i created file /usr/share/nftables.d/chain-post/mangle_prerouting/20-save-mark.nft

meta mark set ct mark 
mark != 0x0 counter accept

iifname "wan_a" ct mark set 0x00000002
iifname "wan_b" ct mark set 0x00000003

ct mark set meta mark

But after applying no access to internet from lan. And it seems marking not working.
Please help me with last step (config nftables)

1 Like
2 
- ct mark set meta mark
+ meta mark set ct mark
1 Like
3 
ct mark != 0x0 meta mark set ct mark return
ct mark set iifname map { "wan_a" : 0x2 , "wan_b" : 0x3 }
meta mark set ct mark return

1 save existing connection mark to route mark
2 add connection mark from input interface
3 save connection mark for freshly classified connection

try to find vyos "cgnat" code - you can learn from it.

IF your router is very weak you can try ct zone (int16) instead.

1 Like
4 
  ct mark 0x0 ct mark set iifname map { "wan_a" : 0x2 , "wan_b" : 0x3 }
  meta mark set ct mark return

no branching

5

Unfortunately it didn't work, still no access from lan.
Maybe it is conflict with rule marking all trafic from lan as 0x2?

6

Obviously every hidden rule does something.

7

I did this rule for routing outgoing packages from lan.
Is is possible in rule

ct mark 0x0 ct mark set iifname map { "wan_a" : 0x2 , "wan_b" : 0x3 }
  meta mark set ct mark return

set dafult behavior if not possible restore mark from conn. For example mark 0x2 by default?

8

you can add another labeller in postrouting like

...mark set iif . oif map { lan . wana : 2 , lan wanb :3 }
...
9

i set in mangle_prerouting

iifname "wan_a" meta mark set meta mark & 0xffff01ff | 0x00000100 return
iifname "wan_b" meta mark set meta mark & 0xffff02ff | 0x00000200 return
ct mark set meta mark & 0x0000ff00 return

why in conntrack i saw mark = 0 ?


udp      17 59 src=client_ip dst=wan_b_ip sport=51244 dport=53 packets=1 bytes=79 src=172.19.0.2 dst=client_ip sport=53 dport=51244 packets=1 bytes=111 mark=0 use=1
10

should be ct mark. try conntrack -E -p udp

11

/usr/share/nftables.d/chain-post/mangle_prerouting/20-save-mark.nft

iifname "pppoe-rtk" ct mark set meta mark & 0xffff01ff | 0x00000100
iifname "wan" ct mark set meta mark & 0xffff02ff | 0x00000200

it works!!! i see

udp      17 58 src=client_ip dst=wan_b sport=35254 dport=53 packets=1 bytes=79 src=172.19.0.2 dst=client_ip sport=53 dport=35254 packets=1 bytes=111 mark=512 use=1

i restore mark in /usr/share/nftables.d/chain-post/mangle_output/20-restore-mark.nft
with

ct mark != 0 meta mark set ct mark

and now ping and traceroute work correctly to both wans

But if i request server on openwrt it not work=(
Maybe i should use another table for restore mark?

12

My server works as docker container

13

I give up. Run local nginx as "ingress"

14

I get it works!!!!!

/usr/share/nftables.d/chain-post/mangle_prerouting/20-save-mark.nft

ct mark != 0 meta mark set ct mark return //Restore on packets from docker
iifname "wan_a" ct mark set meta mark & 0xffff01ff | 0x00000100
iifname "wan_b" ct mark set meta mark & 0xffff02ff | 0x00000200

/usr/share/nftables.d/chain-post/mangle_output/40-restore-mark.nft

ct mark != 0 meta mark set ct mark //Restore on local packets
15

Dockers do output and input. You can have port access list style filter inside them.

16

Sorry, I didn't understand (I just learned how a firewall works today)

17

Docker is a separate network namespace - ie other host bolted on next to ours with separate firewall.