Redirecting traffic from LAN addressed to a specific router WAN IP. Then forward traffic to a specific IP/PORT on the LAN

dandreoletti · December 29, 2023, 7:29am

Hi,

Here is the network topology:

(scroll to the right for the full picture )

        /--- Phone
       |.     x.x.x.x                                      ------ Laptop
       |                                                  /       192.168.3.5
Internet <----> Modem GPON <----wan-> OpenWRT Router. <-lan>
               (Bridge Mode)          PPPoE: 1.160.x.x      
                                                           \
                                                            \
                                                             ------- Server
                                                                      192.168.3.10
                                                                       - service:8000

Here is the network resource setup:

Server hosts service on port 8000
service must be accessible via http(s)://service.com from WAN
service must be accessible via http(s)://service.com from LAN users with the same domain name/port
http(s)://service.com's DNS A record set to the pppoe-wan interface IP: 1.160.x.x
OpenWRT runs on 23.05.x

Here is what can reach http(s)://service.com:

Phone can reach service via http(s)://service.com from the wan side.
- How:
  - (a) OpenWRT configured to forwarding traffic using DNAT (IP + Port mapping) from one zone to another.

Here is what should also reach http(s)://service.com:

Laptop should reach http(s)://service.com from the lan side
- How:
  - (b) forward LAN traffic to 1.160.x.x:80/443 in such way it appears to come from WAN ?
  - (a) use the existing forwarding traffic rules used by the phone's traffic to DNAT to LAN

Approache(s) discarded:

(s0) router's dns redefines http(s)://service.com on the lan side - port is not remapped and OpenWRT's luci listen on 80/443 on 'br-lan'

I asking for your guidance on how to do (b) or a different approach (hairpin / NAT reflection).

lleachii · December 29, 2023, 7:34am

Have you tried this firewall setting on the web GUI?

screen664

It's under the advanced tab of your Port Forward rule.

dandreoletti · December 29, 2023, 7:51am

Here is the current Port Forward rule for https:

Name: "WAN:443" -> "LAN:8000"

General Settings:
- Restrict to address family: automatic
- Protocol: TCP / UDP
- Source zone: wan
- External port: 443
- Destination zone: lan
- Internal IP address: 192.168.3.10
- Internal port: 8000
Advanced Settings
- Use ipset: (empty)
- Source MAC address: (--add MAC--)
- Source IP address: (any)
- Source port: (any)
- External IP address: (any)
- Enable NAT Loopback: enabled
- Loopback source IP: Use internal IP address
- Reflection zones: (unspecified)
- Match helper: (any)
- Match mark: (empty)
- Limit matching: (unlimited)

Enable NAT loopback is already enabled and 'Loopback source IP' is already set to Use Internal IP address.

With these settings, http(s)://service.com is not reachable from LAN (yet ).

lleachii · December 29, 2023, 7:54am

You'd need to specify this.

dandreoletti · December 29, 2023, 8:05am

The following settings were changed as follow:

Reflection zones: lan, wan
- With these setting, http(s)://service.com is not reachable from LAN.
Reflection zones: lan
- With this settings, http(s)://service.com is not reachable from LAN.

Firewall lan zone configured as follow:

PS: I don't understand what reflection zone is and how it works. OpenWRT official doc has limited info.

pavelgl · December 29, 2023, 8:46am

The reflection zone is the internal zone from which the service must be reached using the public IP address.

If NAT Loopback is enabled and the reflection zone is not specified, rules will be created based on the destination (in your case lan) zone specified in the initial DNAT rule. This means that only lan clients will be able to access the service via the public IP.

If you have a guest network and want the guest clients to also access the service via public IP, you need to add the guest zone to the Reflection Zones.

Verify that the rules were created using the CLI.

Run nft list chain inet fw4 dstnat_lan and look for

ip saddr 192.168.3.0/24 ip daddr 1.160.x.x tcp dport 443 dnat ip to 192.168.3.10:8000

Run nft list chain inet fw4 srcnat_lan and look for

ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 snat ip to 192.168.3.1

dandreoletti · December 29, 2023, 9:25am

Got it now. thank you.

Found it, albeit daddr contains both 1.160.x.x and another IP: 192.168.1.106.

PS: With 2 IP in daddr, I tried removing the 192.168.1.106 IP from this rule and the status quo did not change: http(s)://service.com is not reachable from LAN .

ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } tcp dport 443 dnat ip to 192.168.3.10:8000 comment "!fw4: https proxy vm (reflection)"

Note: 192.168.1.106 is the DHCP assigned IP for another interface (DHCP client) named modem, used to access Modem GPON.

Found it too.

ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 snat ip to 192.168.3.1 comment "!fw4: https proxy vm (reflection)"

pavelgl · December 29, 2023, 9:42am

The second IP is not a problem.

Edit:
I don't know how you access the GPON, but all http/https requests for 192.168.1.106 will be redirected to 192.168.3.10.

Where did that come from?
Based on your previous posts, the port should be 8000 as in the SNAT rule

dandreoletti · December 29, 2023, 11:49am

Modem GPON is accessed via http://192.168.1.1, IP 192.168.1.106 is assigned to the OpenWRT router and used to modify the MODEM settings via its web interface.

You are correct. I copy/pasted the wrong nft rule.

The expected rule (below) is present.

ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } tcp dport 443 dnat ip to 192.168.3.10:8000 comment "!fw4: https proxy vm (reflection)"

PS: I corrected my comment with the right port 8000

pavelgl · December 29, 2023, 12:24pm

OK, my mistake.

If there is such a rule that precedes the correct one, it will never work.

For testing, manually create the necessary rules in the main dstnat and srcnat chains, setting the public IP accordingly.

nft insert rule inet fw4 dstnat ip saddr 192.168.3.0/24 ip daddr 1.160.x.x tcp dport 443 counter dnat ip to 192.168.3.10:8000
nft insert rule inet fw4 srcnat ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 counter snat ip to 192.168.3.1

These rules have counters, so check them for hits by running:

nft list chain inet fw4 dstnat; nft list chain inet fw4 srcnat

If there are hits on both rules but it still doesn't work, you should run tcpdump or wireshark on 192.168.3.10 to see what's going on.

dandreoletti · December 29, 2023, 12:57pm

Here is the dstnat_lan chain at the moment (from nft list chain inet fw4 dstnat_lan):

table inet fw4 {
        chain dstnat_lan {
                ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } tcp dport 80 dnat ip to 192.168.3.10:7999 comment "!fw4: http proxy vm (reflection)"
                ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } udp dport 80 dnat ip to 192.168.3.10:7999 comment "!fw4: http proxy vm (reflection)"
                ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } tcp dport 443 dnat ip to 192.168.3.10:8000 comment "!fw4: https proxy vm (reflection)"
                ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } udp dport 443 dnat ip to 192.168.3.10:8000 comment "!fw4: https proxy vm (reflection)"
                ip saddr 192.168.3.0/24 ip daddr { 1.160.edited.x, 192.168.1.106 } tcp dport 10000 dnat ip to 192.168.3.10:8001 comment "!fw4: ssh proxy vm (gitlab) (reflection)"
        }
}

Let me know if the 2 nft insert rules need to added + counter checked.

pavelgl · December 29, 2023, 1:36pm

Looks good, but these rules don't have counters, so please run the suggested commands and check for hits.

Also for http:

nft insert rule inet fw4 dstnat ip saddr 192.168.3.0/24 ip daddr 1.160.x.x tcp dport 80 counter dnat ip to 192.168.3.10:7999
nft insert rule inet fw4 srcnat ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 7999 counter snat ip to 192.168.3.1

dandreoletti · December 30, 2023, 3:16am

pavelgl:

For testing, manually create the necessary rules in the main dstnat and srcnat chains, setting the public IP accordingly.
nft insert rule inet fw4 dstnat ip saddr 192.168.3.0/24 ip daddr 1.160.x.x tcp dport 443 counter dnat ip to 192.168.3.10:8000
nft insert rule inet fw4 srcnat ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 counter snat ip to 192.168.3.1
These rules have counters, so check them for hits by running:
nft list chain inet fw4 dstnat; nft list chain inet fw4 srcnat

I ran:

wget http://service.com
wget https://service.com

then gathered all 4 'counter' enabled rules output:

cat <(nft list chain inet fw4 dstnat; nft list chain inet fw4 srcnat) | grep "table\|chain\|counter"

table inet fw4 {
        chain dstnat {
                ip saddr 192.168.3.0/24 ip daddr 1.160.edited.x tcp dport 80 counter packets 6 bytes 384 dnat ip to 192.168.3.10:7999
                ip saddr 192.168.3.0/24 ip daddr 1.160.edited.x tcp dport 443 counter packets 286 bytes 17224 dnat ip to 192.168.3.10:8000
table inet fw4 {
        chain srcnat {
                ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 7999 counter packets 0 bytes 0 snat ip to 192.168.3.1
                ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 counter packets 0 bytes 0 snat ip to 192.168.3.1

pavelgl · December 30, 2023, 9:08am

dandreoletti:

ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 7999 counter packets 0 bytes 0 snat ip to 192.168.3.1
ip saddr 192.168.3.0/24 ip daddr 192.168.3.10 tcp dport 8000 counter packets 0 bytes 0 snat ip to 192.168.3.1

The packets do not reach the srcnat chain.
The most likely reason would be the presence of some prohibitive rule(s) in the forward chain.
Also, you're not using nftables and iptables at the same time, are you?

dandreoletti · December 30, 2023, 11:41am

We already established:

DNAT / port forwarding is working from WAN to LAN.
- Associated hairpin settings look correct.
  - I am expecting these settings to not be an issue (for now)

There is no prohibiting rules forward chain (except the OpenWRT's defaults policy)

$ nft list ruleset | grep forward | grep -v "LAN_"

        chain forward {
                type filter hook forward priority filter; policy drop;
                ct state established,related accept comment "!fw4: Allow forwarded established and related flows"
                iifname "br-lan" jump forward_lan comment "!fw4: Handle lan IPv4/IPv6 forward traffic"
                iifname { "eth0", "pppoe-wan" } jump forward_wan comment "!fw4: Handle wan IPv4/IPv6 forward traffic"
        chain forward_lan {
                jump accept_to_wan comment "!fw4: Accept lan to wan forwarding"
                ct status dnat accept comment "!fw4: Accept port forwards"
        chain forward_wan {
                ct status dnat accept comment "!fw4: Accept port forwards"
                ct status dnat accept comment "!fw4: Accept port forwards"
                ct status dnat accept comment "!fw4: Accept port forwards"
        chain mangle_forward {
                type filter hook forward priority mangle; policy accept;

The router defines other zones named LAN_xxx such as for example LAN_IOT:

Each zone is in a different VLAN ID. That's why I removed them from the command output
lan has the "default" VLAN ID and the forward ruleset for it is displayed.

iptables (legacy) is installed, yet I don't recall explicitly installing it.

Both iptables binary version are the same.

$ iptables --version
iptables v1.8.8 (legacy)

$ iptables-legacy --version
iptables v1.8.8 (legacy)

Here are iptables binaries list.

$ ip<Tab>

ip                        ip6tables-legacy-restore  ip6tables-save            iptables                  iptables-legacy-save
ip6tables                 ip6tables-legacy-save     ipcalc.sh                 iptables-legacy           iptables-restore
ip6tables-legacy          ip6tables-restore         iperf3                    iptables-legacy-restore   iptables-save

$ ls -alh /usr/sbin/iptables

lrwxrwxrwx    1 root     root          30 Nov 14 21:38 /usr/sbin/iptables -> /usr/sbin/xtables-legacy-multi

$ ls -alh /usr/sbin/iptables-legacy
lrwxrwxrwx    1 root     root          20 Nov 14 21:38 /usr/sbin/iptables-legacy -> xtables-legacy-multi

xtables-legacy-multi is installed by package iptables-zz-legacy (commit)

I am not quite sure if the command below show package the user installed explicitly.

$opkg list-installed | grep "iptables"

ptables-mod-extra - 1.8.8-2
iptables-mod-ipopt - 1.8.8-2
iptables-zz-legacy - 1.8.8-2

Here are the packages depending on iptables instead of the nftables package, along with the status per package found

$ grep -E "Depends" /usr/lib/opkg/info/*.control | grep "iptables" | grep -v ".*iptables-mod-.*:.*"

> User installed packages.

/usr/lib/opkg/info/dockerd.control:Depends: libc, ca-certificates, containerd, iptables, iptables-mod-extra, ip6tables, kmod-ipt-nat6, libseccomp, kmod-ipt-nat, kmod-ipt-physdev, kmod-nf-ipvs, kmod-veth, tini, uci-firewall.
/usr/lib/opkg/info/fwknopd.control:Depends: libc, iptables, libfko, libpcap1

> Not installed manually:

/usr/lib/opkg/info/iptables-zz-legacy.control:Depends: libc, xtables-legacy


> Installed via luci-app-sqm:

/usr/lib/opkg/info/sqm-scripts.control:Depends: libc, tc, kmod-sched-cake, kmod-ifb, iptables, iptables-mod-ipopt

I uninstalled all packages listed above:

opkg remove fwknopd sqm-scripts dockerd iptables-zz-legacy --force-removal-of-dependent-packages

I rebooted OpenWRT device.

reboot

I can now reach http(s)://service.com from LAN (and reaching from WAN is still working as expected):

wget http(s)://service.com
...
HTTP request sent, awaiting response...
  HTTP/1.1 302 Found

I thank you both @lleachii / @pavelgl for the instructions and intuition / experience to guide me through this.

Now the fun starts: which package (out of 4 uninstalled packages) caused srcnat rules not to be hit.

luci-app-sqm (sqm-scripts): Installed + http(s)://service.com accessible from LAN
dockerd: Installed + http(s)://service.com not accessible from LAN
fknowpd: not tested

system · January 9, 2024, 11:41am

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.