How to internal port-forward on different 'tables'?

ambad4u · January 7, 2025, 3:09pm

greetings to all,

before the question, some info on the 'system' of mine...
i have multiple OpenWRT VMs & have 2isp(s)
there is 1 main OpenWRT (controller of the sorts) & is setup manually w/ static routes/rules
the 'main table' is almost not used
my OpenWRT version is

OpenWrt 24.10.0-rc4, r28211-d55754ce0d

there are several routes/rules/interfaces, but i'll show the ones that are of importance... (i think)

config route
  option target '0.0.0.0'
  option netmask '0.0.0.0'
  option gateway '192.176.1.254'
  option table 'tglobe'
  option interface 'globe'
  
config route
  option target '0.0.0.0/0'
  option gateway '192.176.3.254'
  option table 'tmulti'
  option interface 'multi'

config rule
  option in 'vlan10'
  option priority '1000'
  option lookup 'tglobe'
  
config rule
  option priority '4000'
  option in 'APinternet'
  option lookup 'tglobe'

config rule
  option priority '1000'
  option in 'vlan69'
  option lookup 'tmulti'

config interface 'vlan10'
  option proto 'static'
  option device 'x.10'
  option ipaddr '172.30.10.254'
  option netmask '255.255.255.0'
  option defaultroute '0'
  option ip4table 'tglobe'
  option ip6table 'tglobe'
  option gateway '192.176.1.254'

config interface 'APinternet'
  option proto 'static'
  option device 'x.47'
  option ipaddr '172.30.47.254'
  option netmask '255.255.255.240'
  option defaultroute '0'
  option ip4table 'tglobe'
  option delegate '0'

w/ the redirect below, since both interfaces are using table 'tglobe', i am able to access a 'test' web server that is located in APinternet interface/zone from vlan10 interface/zone (note, all my interface has individual zones)

config redirect
  option dest 'zapinternet'
  option target 'DNAT'
  option name 'test'
  option family 'ipv4'
  list proto 'tcp'
  option src 'zvlan10'
  option src_dport '50080'
  option dest_ip '172.30.47.246'
  option dest_port '80'

the question now is..., how does one perform an internal port forward if the interfaces/zones are in different tables? below is the interface/zone that i would like to have it forward to

config interface 'vlan69'
  option proto 'static'
  option device 'x.69'
  option ipaddr '172.30.69.254'
  option netmask '255.255.255.0'
  option defaultroute '0'
  option ip4table 'tmulti'
  option ip6table 'tmulti'
  option gateway '192.176.3.254'

then i create a redirect (like below) & i believe this is the right one... (although not working yet)
but i am missing something like table flow/route/rule thing...

config redirect
  option dest 'zvlan69'
  option target 'DNAT'
  option name 'test'
  option family 'ipv4'
  list proto 'tcp'
  option src 'zvlan10'
  option src_dport '50080'
  option dest_ip '172.30.47.246'
  option dest_port '80'

may anyone shed light to my predicament?

bonus stuff..., i seem to be using /etc/iproute2/rt_tables for the tables &/or routes/rules, but is auto-magicly converted to nftables?

i thank you for your time reading my post.

krazeh · January 7, 2025, 3:22pm

The same way as you normally would. The firewall and routing are separate.

As it is, you probably need a traffic rule rather than a redirect. The latter is only used when you need to rewrite the source or destination IP address.

ambad4u · January 7, 2025, 3:51pm

i see..., but i am not well versed with routes/rules & such.
all examples found for port forwarding usually/maybe only has 1table for the lookup but w/ my example/scenario is a bit different...

i'm just fairly clueless on how to make it work...

krazeh · January 7, 2025, 4:07pm

The routing comes after the firewall. Firewall rules or redirects just allow matched traffic to pass through and, if necessary, rewrite relevant parts of the header (e.g. destination IP or port).

Once the packet gets passed through to the relevant zone (so after the firewall) it will be routed appropriately.

ambad4u · January 7, 2025, 4:37pm

thank you for the insights..., but i'm still lost right now, my apologies...

this is a problem for people like me trying to dive deeper without learning the basic things...
(even the basic things are rather confusing even for me )

will try to research on nftables

krazeh · January 7, 2025, 4:38pm

If you have limited knowledge and skills I would question why you have such a complicated setup. Is it necessary to have multiple openwrt instances and routing tables/rules?

ambad4u · January 7, 2025, 5:19pm

homelab & trying to learn things... & hoping someone would share their time & knowledge...
since i had 2isp(s), MWAN3 was the option but i opted to have manual things so that i can try to learn...

in regards to my multiple OpenWRT VMs, there are 4instance of them, it may spawn to five
but you are right, its not necessary & darn ugly... but works for me ATM
you may disregard how many i have for now as the question is only for the 'main' one
the others are just a cascade into the main one.

& if time allows, and the lightbulb hits me (hopefully)..., then i'll slowly combine them into 1

so for the info i've gathered..., i may need to:
create a new table/rules for it (from interface/zone vlan10 to vlan69)
OR
merge the tables into 1
OR
something different entirely?

i appreciate your time, thank you.

krazeh · January 7, 2025, 5:24pm

Honestly your best approach would probably be to start over from scratch. I suspect your setup is overly (and unnecessarily) complicated which is going to massively hinder any troubleshooting.

ambad4u · January 8, 2025, 3:43am

apologies...
i really don't know why you are making it look complex/complicated...

i did mention multiple OpenWRT VMs but i never mentioned them as the problem, only the 'main' one.

i have stated a working example, then i stated my problem that is somewhat similar to the example (as a separate table)

my aim is to reach a 'web server' that resides in "interface B" from 'interface A'

so it appears that having 2interfaces w/ separate zones on 2 separate tables are that complicated already...

pavelgl · January 8, 2025, 10:14am

When you play with things like this, you should always check what you've done.

ip route list table tglobe
ip route list table tmulti

If a subnet is not listed in the routing table, the default gateway (if defined) will be used.

I can only speculate based on the snippets you posted, but you probably need to populate both tables correctly to avoid using the gateways:

config route
        option table 'tglobe'
        option target '172.30.69.0/24'
        option interface 'vlan69'

config route
        option table 'tmulti'
        option target '172.30.47.240/28'
        option interface 'APinternet'

...

ambad4u:

then i create a redirect (like below) & i believe this is the right one...

ambad4u:

config redirect
  option dest 'zvlan69'
  option target 'DNAT'
  option name 'test'
  option family 'ipv4'
  list proto 'tcp'
  option src 'zvlan10'
  option src_dport '50080'
  option dest_ip '172.30.47.246'
  option dest_port '80'

The destination zone looks wrong.

ambad4u:

config interface 'vlan69'
  option proto 'static'
  option device 'x.69'
  option ipaddr '172.30.69.254'
  option netmask '255.255.255.0'
  option defaultroute '0'
  option ip4table 'tmulti'
  option ip6table 'tmulti'
  option gateway '192.176.3.254'

When you define a gateway for an interface, it must be on the same subnet.

ambad4u · January 8, 2025, 2:17pm

with the working example & when i do: 'ip route list table tglobe', i get below:

root@OpenWRT-x86:~# ip route list table tglobe
default via 192.176.1.254 dev eth1 proto static 
172.30.10.0/24 dev x.10 proto static scope link 
172.30.47.240/28 dev x.47 proto static scope link 
192.176.1.248/29 dev eth1 proto static scope link

should i create a separate table just for the interaction for vlan10 & vlan69?
would you need more info?
am still trying to research as much as i can.

thank you.

ambad4u · January 8, 2025, 3:03pm

so if i create a table ttest
& if i issue: 'ip route list table ttest', it should contain something like below?

172.30.10.0/24 dev x.10 proto static scope link 
172.30.69.0/24 dev x.69 proto static scope link

am still trying to see how i can do this...

krazeh · January 8, 2025, 3:40pm

I think you're doing that all by yourself. The solution to your problem is unlikely to be creating even more complexity within your setup.

What is the purpose of the multiple routing tables? And zones? Are they actually necessary for the setup you have? Do you have enough knowledge/experience to even be able to confidently answer those questions?

ambad4u · January 9, 2025, 5:44am

i cannot prove anything..., thus reaching out to this forum for insights...

i believe i have clearly stated my problem/scenario... internal port forwarding, but with different tables & i have placed a working example as well...

i'm trying to be as concise & as short as possible w/ the details as people have their own lives & time is priceless... & if i don't get answers, then bummer for me...

on the other hand, you are deviating far beyond the scope of this post...
please man..., let's stop this.

lleachii · January 9, 2025, 11:17am

I'm sure this was answered already - port forwarding and traffic rules are the same, regardless of the number of tables used.

Understanding your use case for tables, not adding default routing on some networks (but specify a gateway), etc. may better clarify so we can assist you better.

This statement is inaccurate and confusing.

you must use rt_tables when you decided to give your routing tables a name (i.e., tglobe)
routing rules describe what conditions, etc. to use a table (e.g. in interface, by DST IP, etc.)
nftables (firewall) is unrelated to the routes and rules configuration

Lastly, I observed some major issues:

ambad4u:

config interface 'vlan10'
  option proto 'static'
  option device 'x.10'
  option ipaddr '172.30.10.254'
  option netmask '255.255.255.0'
  option defaultroute '0'
  option ip4table 'tglobe'
  option ip6table 'tglobe'
  option gateway '192.176.1.254'

Your gateway is invalid here. Gateways must exist within the same subnet.
You have no interface named 'globe'

Aside from another incorrect gateway, you have no interface name 'multi'
Most importantly, these gateway networks/IPs don't appear to exist anywhere on the device

It would really help to explain why you made these complex, redundant - but invalid configs on your main OpenWrt.

ambad4u · January 11, 2025, 8:26pm

this is somewhat a continuation of my previous post(s): (Yet Another Dual/Multi ISP Question)...

i learned a bit on MWAN3 on assigning a specific interface to a specific wan/isp on this post: (Mwan3 rule to specific interface), yet i decided to stick to 'manual' mode for now for the sake of 'learning'...

i have a number of interfaces to serve my network..., some use isp-A exclusively, others use isp-B, while a few wanting whatever is available, thus, a 'failover instance' is added to the mix, lets just assume this is isp-X (yet another OpenWRT instance utilizing MWAN3 failover)

also, each 'lan' interface(s) are not allowed to talk to each other (unless otherwise specified)

so my source interface are as follows:
'globe' interface (wan-A / isp-A) assigned to table tglobe
'pldt' interface (wan-B / isp-B) assigned to table tpldt
'multi' interface (wan-X / isp-X) assigned to table tmulti
all the above interface does not utilize "Use default gateway", thus, the main table is not used

this is how i utilized the tables with what i've learned from the links i posted..., created for each individual 'wan' that i have
(is it possible to use 1 table for 2 or more wan/gateway?)

since i also mentioned multiple OpenWRT instance, we'll focus on the "main" one, the others are irrelevant

noted on this, please disregard that particular phrase for now, thank you for the info!

i am unsure why it is working, it might be with the 'clone' feature or such, but unsure...
i now removed the gateway part in vlan10, rebooted both OpenWRT device & PC & gladly, internet still works.

below is the full config

/etc/config/network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd60:e67d:3640::/48'
	option packet_steering '2'

config device
	option type 'bridge'
	option name 'x'
	list ports 'eth0'
	list ports 'eth4'
	list ports 'eth5'

config bridge-vlan
	option device 'x'
	option vlan '1'
	list ports 'eth0:u*'
	list ports 'eth4:u*'
	list ports 'eth5:u*'

config interface 'management'
	option proto 'static'
	option device 'x.1'
	option ipaddr '172.30.1.254'
	option netmask '255.255.255.0'
	option defaultroute '0'
	option delegate '0'

config interface 'globe'
	option proto 'dhcp'
	option device 'eth1'
	option defaultroute '0'
	option ip4table 'tglobe'
	option ip6table 'tglobe'

config bridge-vlan
	option device 'x'
	option vlan '10'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'vlan10'
	option proto 'static'
	option device 'x.10'
	option ipaddr '172.30.10.254'
	option netmask '255.255.255.0'
	option defaultroute '0'
	option ip4table 'tglobe'
	option ip6table 'tglobe'

config route
	option target '0.0.0.0'
	option netmask '0.0.0.0'
	option gateway '192.176.1.254'
	option table 'tglobe'
	option interface 'globe'

config route
	option target '0.0.0.0'
	option netmask '0.0.0.0'
	option gateway '192.176.1.254'
	option table 'tw1globe'
	option interface 'globe'

config rule
	option in 'vlan71'
	option priority '999'
	option lookup 'tw1globe'

config rule
	option in 'vlan10'
	option priority '1000'
	option lookup 'tglobe'

config bridge-vlan
	option device 'x'
	option vlan '47'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'APinternet'
	option proto 'static'
	option device 'x.47'
	option ipaddr '172.30.47.254'
	option netmask '255.255.255.240'
	option defaultroute '0'
	option ip4table 'tglobe'
	option delegate '0'

config rule
	option priority '999'
	option in 'vlan72'
	option lookup 'tw1pldt'

config rule
	option priority '1000'
	option in 'vlan20'
	option lookup 'tpldt'

config rule
	option priority '1000'
	option in 'vlan69'
	option lookup 'tmulti'

config rule
	option priority '4000'
	option in 'APinternet'
	option lookup 'tglobe'

config interface 'pldt'
	option proto 'dhcp'
	option device 'eth2'
	option defaultroute '0'
	option ip4table 'tpldt'
	option ip6table 'tpldt'

config bridge-vlan
	option device 'x'
	option vlan '20'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'vlan20'
	option proto 'static'
	option device 'x.20'
	option ipaddr '172.30.20.254'
	option netmask '255.255.255.0'
	option defaultroute '0'
	option ip4table 'tpldt'
	option ip6table 'tpldt'

config route
	option interface 'pldt'
	option target '0.0.0.0/0'
	option gateway '192.176.2.254'
	option table 'tpldt'

config route
	option interface 'pldt'
	option target '0.0.0.0/0'
	option gateway '192.176.2.254'
	option table 'tw1pldt'

config bridge-vlan
	option device 'x'
	option vlan '69'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'vlan69'
	option proto 'static'
	option device 'x.69'
	option ipaddr '172.30.69.254'
	option netmask '255.255.255.0'
	option defaultroute '0'
	option ip4table 'tmulti'
	option ip6table 'tmulti'

config bridge-vlan
	option device 'x'
	option vlan '99'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config bridge-vlan
	option device 'x'
	option vlan '86'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'vlan86'
	option proto 'static'
	option device 'x.86'
	option ipaddr '172.30.86.254'
	option netmask '255.255.255.240'
	option defaultroute '0'
	option delegate '0'

config bridge-vlan
	option device 'x'
	option vlan '71'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config bridge-vlan
	option device 'x'
	option vlan '72'
	list ports 'eth0:t'
	list ports 'eth4:t'
	list ports 'eth5:t'

config interface 'vlan71'
	option proto 'static'
	option device 'x.71'
	option ipaddr '172.30.71.254'
	option netmask '255.255.255.248'
	option defaultroute '0'
	option ip4table 'tw1globe'
	option ip6table 'tw1globe'
	option delegate '0'

config interface 'vlan72'
	option proto 'static'
	option device 'x.72'
	option ipaddr '172.30.72.254'
	option netmask '255.255.255.248'
	option defaultroute '0'
	option ip4table 'tw1pldt'
	option ip6table 'tw1pldt'
	option delegate '0'

config interface 'multi'
	option proto 'dhcp'
	option device 'eth3'
	option defaultroute '0'
	option ip4table 'tmulti'
	option ip6table 'tmulti'

config rule
	option in 'vlan10'
	option lookup 'ttest'
	option priority '5000'

config rule
	option in 'vlan69'
	option lookup 'ttest'
	option priority '5000'

config route
	option interface 'multi'
	option target '0.0.0.0/0'
	option gateway '192.176.3.254'
	option table 'tmulti'

/etc/config/firewall

config defaults
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option name 'lan'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'

config forwarding
	option src 'lan'
	option dest 'wan'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option dest 'lan'
	option proto 'esp'
	option target 'ACCEPT'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest 'lan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'

config zone
	option name 'zvlan20'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan20'

config zone
	option name 'zvlan72'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan72'

config zone
	option name 'zvlan10'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan10'

config zone
	option name 'zvlan71'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan71'

config zone
	option name 'zapinternet'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'APinternet'

config zone
	option name 'zvlan69'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan69'

config zone
	option name 'zmanagement'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'management'

config zone
	option name 'zglobe'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'globe'

config forwarding
	option src 'zvlan10'
	option dest 'zglobe'

config zone
	option name 'zpldt'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'pldt'

config forwarding
	option src 'zapinternet'
	option dest 'zglobe'

config forwarding
	option src 'zvlan20'
	option dest 'zpldt'

config zone
	option name 'zmulti'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'multi'

config zone
	option name 'zvlan86'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'vlan86'

config zone
	option name 'zvlan99'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'

config forwarding
	option src 'zvlan71'
	option dest 'zglobe'

config forwarding
	option src 'zvlan72'
	option dest 'zpldt'

config forwarding
	option src 'zvlan69'
	option dest 'zmulti'

config rule
	option name 'test'
	option family 'ipv4'
	list proto 'tcp'
	option src 'zvlan10'
	option src_port '50080'
	option dest 'zapinternet'
	option dest_port '80'
	option target 'ACCEPT'
	list dest_ip '172.30.47.246'
	option enabled '0'

config redirect
	option dest 'zvlan69'
	option target 'DNAT'
	option name 'test'
	option family 'ipv4'
	list proto 'tcp'
	option src 'zvlan10'
	option src_dport '50080'
	option dest_ip '172.30.69.252'
	option dest_port '80'

to circle back..., i have a test web server located in 'APinternet' & is accessible via 'vlan10' since i think both intarface is referencing the 'tglobe' table (& w/ the help of the static route/rule)....

now, if i try to move the test web server to 'vlan69', i am not able to access it from 'vlan10' since 'vlan69' uses 'tmulti' table & 'vlan10' is using 'tglobe' table