Port forwarding won't work: rules not transcribed from UCI into iptables (OpenWrt 18.06.4)

Installing and Using OpenWrt Network and Wireless Configuration
1

I'm having problems getting port forwarding to work with OpenWrt 18.06.4. I've created a forwarding rule in LUCI, and the rule appears to be correctly transcribed into /etc/config/firewall, but the rule is getting lost somewhere deeper in the configuration system and seemingly isn't being loaded into iptables.
Manually reloading the configuration with uci commit firewall & reload_config & /etc/init.d/firewall restart does not help.

/etc/init.d/firewall restart shows no errors other than being unable to find the ipset utility.

My intent is to forward port 9999 on the external interface (which is a PPPoE connection in the WAN firewall zone) to port 9999 on 192.168.0.1 on the LAN interface. (192.168.0.1 is not the LAN address for OpenWrt)

LUCI created the following rule in /etc/config/firewall:

config redirect
        option target 'DNAT'
        option src 'wan'
        option dest 'lan'
        option proto 'tcp'
        option src_dport '9999'
        option dest_ip '192.168.0.1'
        option dest_port '9999'
        option name 'Server heartbeat'

However, port 9999 remains closed on the WAN interface. Dumping the iptables rules with iptables -S shows no rules for port 9999:

root@OpenWrt:/etc/config# iptables -S
-P INPUT ACCEPT
-P FORWARD DROP
-P OUTPUT ACCEPT
-N forwarding_lan_rule
-N forwarding_rule
-N forwarding_wan_rule
-N input_lan_rule
-N input_rule
-N input_wan_rule
-N output_lan_rule
-N output_rule
-N output_wan_rule
-N reject
-N syn_flood
-N zone_lan_dest_ACCEPT
-N zone_lan_forward
-N zone_lan_input
-N zone_lan_output
-N zone_lan_src_ACCEPT
-N zone_wan_dest_ACCEPT
-N zone_wan_dest_REJECT
-N zone_wan_forward
-N zone_wan_input
-N zone_wan_output
-N zone_wan_src_REJECT
-A INPUT -i lo -m comment --comment "!fw3" -j ACCEPT
-A INPUT -m comment --comment "!fw3: Custom input rule chain" -j input_rule
-A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m comment --comment "!fw3" -j syn_flood
-A INPUT -i br-lan -m comment --comment "!fw3" -j zone_lan_input
-A INPUT -i eth1 -m comment --comment "!fw3" -j zone_wan_input
-A INPUT -i pppoe -m comment --comment "!fw3" -j zone_wan_input
-A FORWARD -m comment --comment "!fw3: Custom forwarding rule chain" -j forwarding_rule
-A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A FORWARD -i br-lan -m comment --comment "!fw3" -j zone_lan_forward
-A FORWARD -i eth1 -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -i pppoe -m comment --comment "!fw3" -j zone_wan_forward
-A FORWARD -m comment --comment "!fw3" -j reject
-A OUTPUT -o lo -m comment --comment "!fw3" -j ACCEPT
-A OUTPUT -m comment --comment "!fw3: Custom output rule chain" -j output_rule
-A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
-A OUTPUT -o br-lan -m comment --comment "!fw3" -j zone_lan_output
-A OUTPUT -o eth1 -m comment --comment "!fw3" -j zone_wan_output
-A OUTPUT -o pppoe -m comment --comment "!fw3" -j zone_wan_output
-A reject -p tcp -m comment --comment "!fw3" -j REJECT --reject-with tcp-reset
-A reject -m comment --comment "!fw3" -j REJECT --reject-with icmp-port-unreachable
-A syn_flood -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 25/sec --limit-burst 50 -m comment --comment "!fw3" -j RETURN
-A syn_flood -m comment --comment "!fw3" -j DROP
-A zone_lan_dest_ACCEPT -o br-lan -m comment --comment "!fw3" -j ACCEPT
-A zone_lan_forward -m comment --comment "!fw3: Custom lan forwarding rule chain" -j forwarding_lan_rule
-A zone_lan_forward -m comment --comment "!fw3: Zone lan to wan forwarding policy" -j zone_wan_dest_ACCEPT
-A zone_lan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
-A zone_lan_forward -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
-A zone_lan_input -m comment --comment "!fw3: Custom lan input rule chain" -j input_lan_rule
-A zone_lan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
-A zone_lan_input -m comment --comment "!fw3" -j zone_lan_src_ACCEPT
-A zone_lan_output -m comment --comment "!fw3: Custom lan output rule chain" -j output_lan_rule
-A zone_lan_output -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
-A zone_lan_src_ACCEPT -i br-lan -m conntrack --ctstate NEW,UNTRACKED -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_ACCEPT -o eth1 -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
-A zone_wan_dest_ACCEPT -o eth1 -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_ACCEPT -o pppoe -m conntrack --ctstate INVALID -m comment --comment "!fw3: Prevent NAT leakage" -j DROP
-A zone_wan_dest_ACCEPT -o pppoe -m comment --comment "!fw3" -j ACCEPT
-A zone_wan_dest_REJECT -o eth1 -m comment --comment "!fw3" -j reject
-A zone_wan_dest_REJECT -o pppoe -m comment --comment "!fw3" -j reject
-A zone_wan_forward -m comment --comment "!fw3: Custom wan forwarding rule chain" -j forwarding_wan_rule
-A zone_wan_forward -p esp -m comment --comment "!fw3: Allow-IPSec-ESP" -j zone_lan_dest_ACCEPT
-A zone_wan_forward -p udp -m udp --dport 500 -m comment --comment "!fw3: Allow-ISAKMP" -j zone_lan_dest_ACCEPT
-A zone_wan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
-A zone_wan_forward -m comment --comment "!fw3" -j zone_wan_dest_REJECT
-A zone_wan_input -m comment --comment "!fw3: Custom wan input rule chain" -j input_wan_rule
-A zone_wan_input -p udp -m udp --dport 68 -m comment --comment "!fw3: Allow-DHCP-Renew" -j ACCEPT
-A zone_wan_input -p icmp -m icmp --icmp-type 8 -m comment --comment "!fw3: Allow-Ping" -j ACCEPT
-A zone_wan_input -p igmp -m comment --comment "!fw3: Allow-IGMP" -j ACCEPT
-A zone_wan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
-A zone_wan_input -m comment --comment "!fw3" -j zone_wan_src_REJECT
-A zone_wan_output -m comment --comment "!fw3: Custom wan output rule chain" -j output_wan_rule
-A zone_wan_output -m comment --comment "!fw3" -j zone_wan_dest_ACCEPT
-A zone_wan_src_REJECT -i eth1 -m comment --comment "!fw3" -j reject
-A zone_wan_src_REJECT -i pppoe -m comment --comment "!fw3" -j reject

I haven't tried upgrading OpenWrt as I'm not seeing anything in the changelogs for .5 or .6 that looks immediately relevant to this issue.

I'm going to try working around this with a custom rule (or, in the worst case, with a script that adds firewall rules by invoking iptables directly) but from where I sit this looks like a bug.

Hardware here is a QEMU/KVM virtual machine running the x86 version of OpenWrt.

2

Please post your complete firewall file cat /etc/config/firewall

Also, have you verified that the server is indeed listening on port 9999 -- i.e. test the connection from within the LAN. Using external port scanning websites can be hit or miss in general, and they cannot differentiate between the lack of response from the server vs port forwarding not configured properly.

3

The server is reachable inside the LAN. It's also reachable externally if I setup iptables manually or put the appropriate forwarding rules in /etc/firewall.user. It's only unreachable externally if I try to do things the easy way (with LuCI) or the easier way (by editing /etc/config/firewall).

Here's the contents of /etc/config/firewall as generated by LuCI.

config redirect
        option target 'DNAT'
        option src 'wan'
        option dest 'lan'
        option proto 'tcp'
        option src_dport '9999'
        option dest_ip '192.168.0.1'
        option dest_port '9999'
        option name 'Server heartbeat'

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option network 'wan wan6 eth1a pppoe'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

Bonus: output from /etc/init.d/firewall restart

root@OpenWrt:/etc/config# /etc/init.d/firewall restart
Warning: Unable to locate ipset utility, disabling ipset support
 * Flushing IPv4 filter table
 * Flushing IPv4 nat table
 * Flushing IPv4 mangle table
 * Flushing IPv6 filter table
 * Flushing IPv6 mangle table
 * Flushing conntrack table ...
 * Populating IPv4 filter table
   * Rule 'Allow-DHCP-Renew'
   * Rule 'Allow-Ping'
   * Rule 'Allow-IGMP'
   * Rule 'Allow-IPSec-ESP'
   * Rule 'Allow-ISAKMP'
   * Redirect 'Server heartbeat'
   * Forward 'lan' -> 'wan'
   * Zone 'lan'
   * Zone 'wan'
 * Populating IPv4 nat table
   * Redirect 'Server heartbeat'
   * Zone 'lan'
   * Zone 'wan'
 * Populating IPv4 mangle table
   * Zone 'lan'
   * Zone 'wan'
 * Populating IPv6 filter table
   * Rule 'Allow-DHCPv6'
   * Rule 'Allow-MLD'
   * Rule 'Allow-ICMPv6-Input'
   * Rule 'Allow-ICMPv6-Forward'
   * Rule 'Allow-IPSec-ESP'
   * Rule 'Allow-ISAKMP'
   * Forward 'lan' -> 'wan'
   * Zone 'lan'
   * Zone 'wan'
 * Populating IPv6 mangle table
   * Zone 'lan'
   * Zone 'wan'
 * Set tcp_ecn to off
 * Set tcp_syncookies to on
 * Set tcp_window_scaling to on
 * Running script '/etc/firewall.user'
4

I wonder if the fact that it is first in the chain is causing the problem. While normally the rules operate in the order they appear, typically redirect rules show up at below all the default stuff. Try moving the forward rule down below the include for the firewall.user.

Also, this one is probably not anything, but I wonder if the number of networks in the wan zone are causing issues -- I would expect to have either wan or pppoe (wan6 if you do have dual stack), but eth1a seems odd to me.

Oh, and I ran the same restart of the firewall on an otherwise default config, and I get the same warning about ipset -- probably doesn't matter.

5

I'll try reordering the rules when I have a moment.

eth1a is a virtual interface I created to access the management interface of my modem from inside the LAN. It's not something OpenWRT created automatically.

6

In my experience, this should not be necessary. Obviously YMMV.
My upstream device is a Arris cable modem that has a private/local IP of 192.168.100.1, but is a true bridge device, providing a public IP address from my ISP to the WAN of my router. My internal network is on a different subnet. I have never needed to create any special rules or interfaces to connect to the modem's status page... I just type the address into my browser and it works.

I have seen many instances of people creating virtual interfaces /VLANs and/or firewall rules that are intended allow upstream connectivity, only to experience issues as a result of these added complications and to discover that it wasn't necessary in the first place. You might just try removing that virtual interface to see if you really need it.

7

The iptables -S command only dumps the filter table, not the nat one. Use iptables -t nat -S to see the port forward rules.

An alternative to see the complete ruleset is running fw3 print or more specifically fw3 print | grep "Server heartbeat" to see the heartbeat related rules.

Your configuration looks correct to me and assuming that the DNAT rules are created properly, the problem would likely be the the routing between your OpenWrt gateway and the target host. In particular make sure that the target host is using the OpenWrt device as gateway and that something is actually listening on port 9999 there.

1 Like
8

Here's the output from fw3 print | grep heartbeat:

iptables -t nat -A zone_wan_prerouting -p tcp -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat" -j DNAT --to-destination 192.168.0.1:9999
iptables -t nat -D zone_lan_prerouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.1.2/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3:Server heartbeat (reflection)" -j DNAT --to-destination 192.168.0.1:9999
iptables -t nat -A zone_lan_prerouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.1.2/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j DNAT --to-destination 192.168.0.1:9999
iptables -t nat -D zone_lan_postrouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.0.1/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j SNAT --to-source 192.168.0.3
iptables -t nat -A zone_lan_postrouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.0.1/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j SNAT --to-source 192.168.0.3
iptables -t nat -D zone_lan_prerouting -p tcp -s 192.168.0.0/255.255.255.0 -d <external IP>/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j DNAT --to-destination 192.168.0.1:9999
iptables -t nat -A zone_lan_prerouting -p tcp -s 192.168.0.0/255.255.255.0 -d <external IP>/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j DNAT --to-destination 192.168.0.1:9999
iptables -t nat -D zone_lan_postrouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.0.1/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j SNAT --to-source 192.168.0.3
iptables -t nat -A zone_lan_postrouting -p tcp -s 192.168.0.0/255.255.255.0 -d 192.168.0.1/255.255.255.255 -m tcp --dport 9999 -m comment --comment "!fw3: Server heartbeat (reflection)" -j SNAT --to-source 192.168.0.3

To reiterate, the target host is reachable from the Internet if I set the following rules in firewall.user:

iptables -A FORWARD -i pppoe -o br-lan -p tcp --syn --dport 9999 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -i pppoe -o br-lan -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -A FORWARD -i br-lan -o pppoe -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -i pppoe -p tcp --dport 9999 -j DNAT --to-destination 192.168.0.1
iptables -t nat -A POSTROUTING -o br-lan -p tcp --dport 9999 -d 192.168.0.1 -j SNAT --to-source 192.168.0.3
9

Apparently you need an extra SNAT rule towards lan to properly reach the host... in this case you need to declare an additional config nat for fw3:

config nat
  option proto tcp
  option src lan
  option dest_ip 192.168.0.1
  option dest_port 9999
  option snat_ip 192.168.0.3
  option target SNAT