Port-Forwarding in a multi VLAN environment

Hello.
I'm currently running into an issue that involves port forwarding. I can reach my services from my mobile phone (LTE, not Wi-Fi) but I cannot connect to them when accessing my public IP over LAN.

It's probably related to reflection because when I set up a MASQUERADING NAT rule for the services, it works (but I completely lost my source origin IP, from the service perspective). I already tried to use reflection_zone but without success.

ISP: 300/90 Fiber (GPON) - PPPoE - Static Public IP without port restrictions

OpenWRT Version: 22.03-rc5
Hardware: FriendlyElec R4S

My environment consists of 3 VLANs/Zones:

  • Management (10.10.0.0/24)
  • Services (10.30.0.0/24)
  • Lan (10.40.0.0/24)

I'm running a reverse proxy on 10.30.0.2 (HTTP and HTTPS).
Configured port forwarding for both:

config redirect
        option dest 'services'
        option target 'DNAT'
        option name 'Services-HTTP'
        option src 'wan'
        option src_dport '80'
        option dest_ip '10.30.0.2'
        option dest_port '80'
        option src_dip '<public IP>'

config redirect
        option dest 'services'
        option target 'DNAT'
        option name 'Services-HTTPS'
        option src 'wan'
        option src_dport '443'
        option dest_ip '10.30.0.2'
        option dest_port '443'
        option src_dip '<public IP>'

In your case FW4 will create NAT loopback rules for the services zone only.
If you initiate a request to your public IP originating from this zone, it should work.
Try creating additional DNAT rules for the lan zone with option reflection set to 0.

config redirect
        option target 'DNAT'
        option name 'Redirect-HTTP'
        option proto 'tcp'
        option src 'lan'
        option dest 'wan'
        option src_dport '80'
        option dest_port '80'
        option reflection '0'
        option dest_ip '10.30.0.2'
        option src_dip '<public IP>'

config redirect
        option target 'DNAT'
        option name 'Redirect-HTTPS'
        option proto 'tcp'
        option src 'lan'
        option dest 'wan'
        option src_dport '443'
        option dest_port '443'
        option reflection '0'
        option dest_ip '10.30.0.2'
        option src_dip '<public IP>'

Check that the rules are created correctly and that there are hits.

nft list chain inet fw4 dstnat_lan

It didn't work. When scanning through tcpdump, it shows that the web server is trying to respond to my computer.

TCP Dump on the server

tcpdump: listening on eth0.30, link-type EN10MB (Ethernet), snapshot length 262144 bytes
19:01:30.447331 IP (tos 0x0, ttl 62, id 6513, offset 0, flags [DF], proto TCP (6), length 60)
    10.40.0.131.1530 > 10.30.0.2.80: Flags [S], cksum 0x6b2c (correct), seq 2924359166, win 64240, options [mss 1460,sackOK,TS val 2057072613 ecr 0,nop,wscale 7], length 0
19:01:30.451588 IP (tos 0x0, ttl 62, id 51871, offset 0, flags [DF], proto TCP (6), length 60)
    10.40.0.131.1534 > 10.30.0.2.80: Flags [S], cksum 0xd825 (correct), seq 3332810916, win 64240, options [mss 1460,sackOK,TS val 2057072617 ecr 0,nop,wscale 7], length 0
19:01:30.551445 IP (tos 0x0, ttl 62, id 40983, offset 0, flags [DF], proto TCP (6), length 60)
    10.40.0.131.1536 > 10.30.0.2.80: Flags [S], cksum 0xda56 (correct), seq 4179391383, win 64240, options [mss 1460,sackOK,TS val 2057072717 ecr 0,nop,wscale 7], length 0

TCP Dump on the server container:

[root@docker-0b9be4ce6f7f:/app]# tcpdump -n -vv port 80
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
19:01:30.447440 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    10.30.0.2.80 > 10.40.0.131.1530: Flags [S.], cksum 0x14f9 (incorrect -> 0x243d), seq 3337747558, ack 2924359167, win 65160, options [mss 1460,sackOK,TS val 2416242664 ecr 2057072613,nop,wscale 7], length 0
19:01:30.451801 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    10.30.0.2.80 > 10.40.0.131.1534: Flags [S.], cksum 0x14f9 (incorrect -> 0x9034), seq 443853282, ack 3332810917, win 65160, options [mss 1460,sackOK,TS val 2416242668 ecr 2057072617,nop,wscale 7], length 0
19:01:30.551586 IP (tos 0x0, ttl 64, id 0, offset 0, flags [DF], proto TCP (6), length 60)
    10.30.0.2.80 > 10.40.0.131.1536: Flags [S.], cksum 0x14f9 (incorrect -> 0xcc63), seq 3480664701, ack 4179391384, win 65160, options [mss 1460,sackOK,TS val 2416242768 ecr 2057072717,nop,wscale 7], length 0