Device as router with disabled NAT, additional routing rules

Livy · December 16, 2019, 3:35pm

I need some help configuring the firewall and static routes. I am trying what is described here. My topology:

/etc/config/network

config interface 'loopback'
	option ifname 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix 'fd73:c662:4d27::/48'

config interface 'lan'
	option type 'bridge'
	option ifname 'eth0.1'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option ipaddr '192.168.100.1'

config device 'lan_dev'
	option name 'eth0.1'
	option macaddr '20:76:93:46:64:4f'

config interface 'wan'
	option ifname 'eth0.2'
	option proto 'dhcp'

config device 'wan_dev'
	option name 'eth0.2'
	option macaddr '20:76:93:46:64:50'

config interface 'wan6'
	option ifname 'eth0.2'
	option proto 'dhcpv6'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option ports '0 1 2 3 6t'

config switch_vlan
	option device 'switch0'
	option vlan '2'
	option ports '4 6t'

# This is what I added:
config route
	option interface 'wan'
	option target '192.168.1.0'
	option netmask '255.255.255.0'

config route
	option interface 'wan'
	option target '0.0.0.0'
	option netmask '0.0.0.0'
	option gateway '192.168.1.1'

/etc/config/firewall

config defaults
	option syn_flood	1
	option input		ACCEPT
	option output		ACCEPT
	option forward		REJECT

config zone
	option name		lan
	list   network		'lan'
	option input		ACCEPT
	option output		ACCEPT
	option forward		ACCEPT

# input changed from REJECT to ACCEPT, masq changed from 1 to 0
config zone
	option name		wan
	list   network		'wan'
	list   network		'wan6'
	option input		ACCEPT
	option output		ACCEPT
	option forward		REJECT
	option masq		0
	option mtu_fix		1

config forwarding
	option src		lan
	option dest		wan

# This is what I added:
config forwarding
	option src		wan
	option dest		lan

config rule
	option name		Allow-DHCP-Renew
	option src		wan
	option proto		udp
	option dest_port	68
	option target		ACCEPT
	option family		ipv4

config rule
	option name		Allow-Ping
	option src		wan
	option proto		icmp
	option icmp_type	echo-request
	option family		ipv4
	option target		ACCEPT

config rule
	option name		Allow-IGMP
	option src		wan
	option proto		igmp
	option family		ipv4
	option target		ACCEPT

config rule
	option name		Allow-DHCPv6
	option src		wan
	option proto		udp
	option src_ip		fc00::/6
	option dest_ip		fc00::/6
	option dest_port	546
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-MLD
	option src		wan
	option proto		icmp
	option src_ip		fe80::/10
	list icmp_type		'130/0'
	list icmp_type		'131/0'
	list icmp_type		'132/0'
	list icmp_type		'143/0'
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-ICMPv6-Input
	option src		wan
	option proto	icmp
	list icmp_type		echo-request
	list icmp_type		echo-reply
	list icmp_type		destination-unreachable
	list icmp_type		packet-too-big
	list icmp_type		time-exceeded
	list icmp_type		bad-header
	list icmp_type		unknown-header-type
	list icmp_type		router-solicitation
	list icmp_type		neighbour-solicitation
	list icmp_type		router-advertisement
	list icmp_type		neighbour-advertisement
	option limit		1000/sec
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-ICMPv6-Forward
	option src		wan
	option dest		*
	option proto		icmp
	list icmp_type		echo-request
	list icmp_type		echo-reply
	list icmp_type		destination-unreachable
	list icmp_type		packet-too-big
	list icmp_type		time-exceeded
	list icmp_type		bad-header
	list icmp_type		unknown-header-type
	option limit		1000/sec
	option family		ipv6
	option target		ACCEPT

config rule
	option name		Allow-IPSec-ESP
	option src		wan
	option dest		lan
	option proto		esp
	option target		ACCEPT

config rule
	option name		Allow-ISAKMP
	option src		wan
	option dest		lan
	option dest_port	500
	option proto		udp
	option target		ACCEPT

config include
	option path /etc/firewall.user

First question: the strange thing I encountered is that, if I change wan option masq to 0 to disable SNAT, I can no longer access the Internet. A simple tracert shows that the traffic reaches the ISP router and stops there.

PS C:\Users\livyi> tracert facebook.com
Tracing route to facebook.com [31.13.95.36] over a maximum of 30 hops:
1    <1 ms    <1 ms    <1 ms  OpenWrt.lan [192.168.100.1]
2    <1 ms    <1 ms    <1 ms  192.168.1.1
3     *        *        *     Request timed out.

Maybe the ISP router denies doing NAT for client IPs which is out of its subnet range 192.168.1.0/24?

Second question: what did I do wrong in the config above? I can ping PC1 from PC2, but not to PC2 from PC1 -- I want to make both PCs access each other. The static route was added to the ISP router to forward 192.168.100.0/24 traffic to gateway 192.168.1.41. And I can ping 192.168.100.1 from PC1.

AndrewZ · December 16, 2019, 3:50pm

Of course. If you can change the configuration of the ISP router, just change the mask on the LAN port from /24 to /16.

Livy · December 16, 2019, 4:59pm

It's nice to know that. However, after I followed your suggestions, a new problem arose: PC1 could no longer connect to PC2, because it tried to search for 192.168.100.172 inside its own broadcast domain (they are all /16), instead of contacting the OpenWrt router at 192.168.1.41. Adding the static route to the OpenWrt router on every host of the subnet 192.168.0.0/16 is not really convenient.

lleachii · December 16, 2019, 5:07pm

That's normal. You are sending traffic to your ISP's router LAN side with a SRC address of 192.168.100.x. It's behaving as a normal router would. You disabled masquerade on the OpenWrt.

You didn't configure NAT, your router can't guess why it's receiving traffic outside the LAN range of 192.168.1.0/24. Nonetheless, the ISP doesn't have a route to return that 192.168.100.0/24 traffic to you - and it shouldn't because they're Private IPs!

PC1 needs a route for 192.168.100.0/24 at gateway 192.168.1.41.

I advise not turning masquerade off until you really understand routing.

Also, two major notes:

you may need to ensure that your WAN firewall setting is FORWARD ENABLED; and
also some routers need a sysctrl settings to forward to another router in the same LAN.

trendy · December 16, 2019, 6:44pm

Your OpenWrt configuration is correct, given your requirements for internet access and internal communication.
The 2 routes you added are not necessary. 192.168.1.0/24 is directly connected and the 0/0 will be acquired from DHCP server of ISP router.

I believe the problem here is the router of the ISP. It needs:

Static route for 192.168.100.0/24 via 192.168.1.41
To Masquerade the 192.168.100.0/24
You have accomplished the first. If the second is not possible due to firmware limitations, then you can select not to Masquerade traffic going to 192.168.1.0/24 on the OpenWrt, but will masquerade the rest.
option masq_dest '!192.168.1.0/24'

Livy · December 16, 2019, 7:37pm

Finally, I found that I had done nothing wrong (yes the 2 static routes were redundant, as they were my desperate attempts). I tried running a web server on PC2, and PC1 could connect to it.

I thought the configuration was wrong because I could not ping PC2 from PC1. Pinging to the OpenWrt router from PC1 was still ok. Why the ping failed? The ICMP packet reached the OpenWrt router and stopped there. Do you see any problem with the firewall above?

trendy · December 16, 2019, 8:20pm

There is no problem,

wrong

but if you apply Masquerade on OpenWrt wan interface and ping an PC2 from PC1, then PC2 reply will be NATed, hence the PC1 will receive a reply with source 192.168.1.41 and will discard it.

Livy · December 17, 2019, 7:44am

Thanks a lot for the masq_dest option. The firmware of my ISP router has no configuration for NAT, and by default it only does NAT for its own subnet. So now, it is (a segment of) my firewall configuration:

# LAN zone settings are default
config zone
	option name		lan
	list   network		'lan'
	option input		ACCEPT
	option output		ACCEPT
	option forward		ACCEPT

config zone
	option name		wan
	list   network		'wan'
	list   network		'wan6'
	option input		ACCEPT
	option output		ACCEPT
	option forward		ACCEPT
	option masq		1
	option masq_dest '!192.168.1.0/24'
	option mtu_fix		1

config forwarding
	option src		lan
	option dest		wan

config forwarding
	option src		wan
	option dest		lan

# This is also default.
config rule
	option name		Allow-Ping
	option src		wan
	option proto		icmp
	option icmp_type	echo-request
	option family		ipv4
	option target		ACCEPT

I also install a web server on both PC1 and PC2. Here is the index.php, which simply shows visitor IP address:

<?php
    //whether ip is from share internet
    if (!empty($_SERVER['HTTP_CLIENT_IP']))   
    {
        echo "Client IP type: shared Internet";
        $ip_address = $_SERVER['HTTP_CLIENT_IP'];
    }
    //whether ip is from proxy
    elseif (!empty($_SERVER['HTTP_X_FORWARDED_FOR']))  
    {
        echo "Client IP type: proxy";
        $ip_address = $_SERVER['HTTP_X_FORWARDED_FOR'];
    }
    //whether ip is from remote address
    else
    {
        echo "Client IP type: remote address (default)";
        $ip_address = $_SERVER['REMOTE_ADDR'];
    }
    echo "</br>Client IP address: $ip_address";
?>

By trying to access the website on PC1 and PC2 from each other, it is confirmed that no NAT takes place between their connections (removing the line option masq_dest '!192.168.1.0/24' does indeed shows the NATed IP 192.168.1.41 when PC2 accesses PC1 website).

Still, the ping from PC2 to PC1 works, while the ping from PC1 to PC2 fails. PC1 is actually an Android phone running Termux. Here is the output of ping command on Termux terminal:

$ ping -c4 192.168.100.172 (192.168.100.172) 56(84) bytes of data.
From 192.168.1.1: icmp_seq=2 Redirect Host (New nexthop: 192.168.1.41)

--- 192.168.100.172 ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3062ms

There is still something wrong here.

Edit: OMG, Bill Gates is the culprit here. My default Windows Firewall settings block all IMCP echo requests. By adding a custom rule to Windows Firewall, it now works. While on the Android phone, ping is enabled by default. I remember the same issue on my workplace several months ago, when some guys called me only because they could not ping the target Windows 10 PC, but when I checked, the connection was working normally. What a frustrating experience -- I wasted a lot of time in the last few days because of this.

mikma · December 17, 2019, 7:49am

Actually the stateful firewall in openwrt takes care of that thanks to conntrack in the kernel. The ICMP echo replies won't be NATed.

Livy · December 17, 2019, 8:11am

One last thing though, before the thread can considered fully solved.

I am going to use OpenWrt as the main router soon, with the ISP configured as bridge mode to act as a dumb modem/ONT. Then, several other OpenWrt routers will be connected to the main one, each with their own subnets and static routes to connect them all together.

How to make the main OpenWrt do SNAT for all subnets? Which configuration should I change?

trendy · December 17, 2019, 8:30am

I could be wrong, I'll try that tonight and let you know.

SNAT is done when you have a static public IP. Then you can define which private IPs are NATed for a specific public IP. https://openwrt.org/docs/guide-user/firewall/firewall_configuration#redirects
Otherwise you can just leave the default masquerade on, which will NAT everything on the egress of the wan.

trendy · December 19, 2019, 11:35pm

@mikma you are right, responses are not NATed.

system · December 29, 2019, 11:35pm

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.