Route does not work with OpenWrt router

qussungtn · October 28, 2019, 9:21am

Dear all,

I want to connect Rasberry pi which is hosting server to WAN port as below. The use of OpenWrt router is to make a command signal delay with netem network emulator.

what I have done

set a proper static IP address for routing.

problem
I can ping both controller(192.168.0.253) and raspberry pi(192.168.1.253) from OpenWrt router. From the raspberry pi, I can ping to controller. It means that there is no problem for routing from WAN TO LAN

However, WAN port(192.168.1.154) is not reachable from controller.

Current ip configuration of router.

Current route-configuration

Current firewall setting

config defaults                                                                                                                 
option syn_flood        1                                                                                               
option input            ACCEPT                                                                                         
option output           ACCEPT                                                                                          
option forward          ACCEPT

config zone                                                                                                                     
option name            lan                                                                                             
list   network          'lan'                                                                                           
option input            ACCEPT                                                                                         
option output         ACCEPT                                                                                          
option forward        ACCEPT

config zone                                                                                                                                                                                                                                                                                                                     option name             wan                                                                                             
list   network            'wan'                                                                                          
list   network            'wan6'                                                                                          
option input             ACCEPT                                                                                         
option output           ACCEPT                                                                                          
option forward         ACCEPT                                                                                          
option masq             1                                                                                               
option mtu_fix          1

config forwarding                                                                                                               
option src              lan                                                                                             
option dest            wan

config rule                                                                                                                     
option name          Allow-DHCP-Renew                                                                               
option src              wan                                                                                             
option proto           udp                                                                                            
option dest_port        68                                                                                             
option target          ACCEPT                                                                                          
option family           ipv4

 Allow IPv4 ping                                                                                                      
config rule                                                                                                                     
option name         Allow-Ping                                                                                      
option src              wan                                                                                             
option proto          icmp                                                                                            
option icmp_type     echo-request                                                                                    
option family           ipv4                                                                                            
option target           ACCEPT

config rule                                                                                                                     
option name          Allow-IGMP                                                                                      
option src              wan               
option proto            igmp                                                                                            
option family           ipv4                                                                                            
option target           ACCEPT

Allow DHCPv6 replies  
config rule                                                                                                                     
option name          Allow-DHCPv6                                                                                    
option src              wan                                                                                             
option proto            udp                                                                                             
option src_ip           fc00::/6                                                                                        
option dest_ip          fc00::/6                                                                                        
option dest_port        546                                                                                             
option family           ipv6                                                                                            
option target           ACCEPT

config rule                                                                                                                   
option name             Allow-MLD                                                                                       
option src              wan                                                                                             
option proto            icmp                                                                                           
option src_ip           fe80::/10                                                                                      
list icmp_type          '130/0'                                                                                         
list icmp_type          '131/0'                                                                                        
list icmp_type          '132/0'                                                                                         
list icmp_type          '143/0'                                                                                         
option family           ipv6                                                                                            
option target           ACCEPT

Allow essential incoming IPv6 ICMP traffic                                                                            
config rule                                                                                                                    
 option name           Allow-ICMPv6-Input                                                                              
option src                wan                                                                                             
option proto           icmp                                                                                                    
list icmp_type          echo-request                                                                                   
 list icmp_type          echo-reply                                                                                      
list icmp_type          destination-unreachable                                                                        
 list icmp_type          packet-too-big                                                                                  
list icmp_type          time-exceeded                                                                                  
 list icmp_type          bad-header                                                                                      
list icmp_type          unknown-header-type                                                                            
 list icmp_type          router-solicitation                                                                             
list icmp_type          neighbour-solicitation                                                                         
 list icmp_type          router-advertisement                                                                           
 list icmp_type          neighbour-advertisement                                                                       
  option limit            1000/sec                                                                                        
option family           ipv6                                                                                            
option target           ACCEPT

 Allow essential forwarded IPv6 ICMP traffic                                                                          
config rule                                                                                                                     
option name           Allow-ICMPv6-Forward                                                                           
option src              wan                                                                                             
option dest             *                                                                                               
option proto            icmp                                                                                           
 list icmp_type          echo-request                                                                                    
list icmp_type          echo-reply                                                                                      
list icmp_type          destination-unreachable    
list icmp_type          packet-too-big                                                                                  
list icmp_type          time-exceeded                                                                                   
list icmp_type          bad-header                                                                                      
list icmp_type          unknown-header-type                                                                             
option limit            1000/sec                                                                                        
option family           ipv6                                                                                            
option target           ACCEPT

config rule                                                                                                                     
option name           Allow-IPSec-ESP                                                                                 
option src                  wan                                                                                             
option dest               lan                                                                                            
 option proto            esp                                                                                             
option target           ACCEPT

config rule                                                                                                                     
option name            Allow-ISAKMP                                                                                    
option src                 wan                                                                                             
option dest               lan                                                                                             
option dest_port       500                                                                                            
option proto            udp                                                                                             
option target           ACCEPT

I need your advice on this problem
please let me know if you need further config information.

Best regards
Byeon Seongsu

anon50098793 · October 28, 2019, 10:43am

if your firewall is really "off" please check your ip_forward setting.... and default policies...

iptables -L
cat /proc/sys/net/ipv4/ip_forward

also show us the route output on both clients / default gateway setting?

( note: if you ever turn the firewall back on.. i don't think you'll want nat on the interface...)

But generally with that topology... client routes and adding a firewall forwarding rule is what most people trip up on. WAN is more complex as unless you want to use it like a STATEFULL interface... you need to undo a few things ( except for the forwarding rule )

qussungtn · October 28, 2019, 10:55am

Thank you for your reply!!

I attached the result from the iptables -L as below.

Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere /* !fw3  <em>/
input_rule all -- anywhere anywhere /</em>  !fw3: Custom input rule chain  <em>/
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED /</em>  !fw3  <em>/
syn_flood tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN /</em>  !fw3  <em>/
zone_lan_input all -- anywhere anywhere /</em>  !fw3  <em>/
zone_wan_input all -- anywhere anywhere /</em>  !fw3  <em>/
Chain FORWARD (policy ACCEPT)
target prot opt source destination
forwarding_rule all -- anywhere anywhere /</em>  !fw3: Custom forwarding rule chain  <em>/
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED /</em>  !fw3  <em>/
zone_lan_forward all -- anywhere anywhere /</em>  !fw3  <em>/
zone_wan_forward all -- anywhere anywhere /</em>  !fw3  <em>/
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
output_rule all -- anywhere anywhere /</em>  !fw3: Custom output rule chain  <em>/
ACCEPT all -- anywhere anywhere ctstate RELATED,ESTABLISHED /</em>  !fw3  <em>/
zone_lan_output all -- anywhere anywhere /</em>  !fw3  <em>/
zone_wan_output all -- anywhere anywhere /</em>  !fw3 */

Chain forwarding_lan_rule (1 references)
target prot opt source destination
Chain forwarding_rule (1 references)
target prot opt source destination
Chain forwarding_wan_rule (1 references)
target prot opt source destination
Chain input_lan_rule (1 references)
target prot opt source destination
Chain input_rule (1 references)
target prot opt source destination
Chain input_wan_rule (1 references)
target prot opt source destination
Chain output_lan_rule (1 references)
target prot opt source destination
Chain output_rule (1 references)
target prot opt source destination
Chain output_wan_rule (1 references)
target prot opt source destination

Chain reject (0 references)
target prot opt source destination
REJECT tcp -- anywhere anywhere /* !fw3  <em>/ reject-with tcp-reset
REJECT all -- anywhere anywhere /</em>  !fw3  <em>/ reject-with icmp-port-unreachable
Chain syn_flood (1 references)
target prot opt source destination
RETURN tcp -- anywhere anywhere tcp flags:FIN,SYN,RST,ACK/SYN limit: avg 25/sec burst 50 /</em> !fw3  <em>/
DROP all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_lan_dest_ACCEPT (4 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_lan_forward (1 references)
target prot opt source destination
forwarding_lan_rule all -- anywhere anywhere /</em>  !fw3: Custom lan forwarding rule chain  <em>/
zone_wan_dest_ACCEPT all -- anywhere anywhere /</em>  !fw3: Zone lan to wan forwarding policy  <em>/
ACCEPT all -- anywhere anywhere ctstate DNAT /</em>  !fw3: Accept port forwards  <em>/
zone_lan_dest_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_lan_input (1 references)
target prot opt source destination
input_lan_rule all -- anywhere anywhere /</em>  !fw3: Custom lan input rule chain  <em>/
ACCEPT all -- anywhere anywhere ctstate DNAT /</em>  !fw3: Accept port redirections  <em>/
zone_lan_src_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_lan_output (1 references)
target prot opt source destination
output_lan_rule all -- anywhere anywhere /</em>  !fw3: Custom lan output rule chain  <em>/
zone_lan_dest_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_lan_src_ACCEPT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate NEW,UNTRACKED /</em>  !fw3  <em>/
Chain zone_wan_dest_ACCEPT (3 references)
target prot opt source destination
DROP all -- anywhere anywhere ctstate INVALID /</em>  !fw3: Prevent NAT leakage  <em>/
ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_wan_forward (1 references)
target prot opt source destination
forwarding_wan_rule all -- anywhere anywhere /</em>  !fw3: Custom wan forwarding rule chain  <em>/
zone_lan_dest_ACCEPT esp -- anywhere anywhere /</em>  !fw3: Allow-IPSec-ESP  <em>/
zone_lan_dest_ACCEPT udp -- anywhere anywhere udp dpt:isakmp /</em>  !fw3: Allow-ISAKMP  <em>/
ACCEPT all -- anywhere anywhere ctstate DNAT /</em>  !fw3: Accept port forwards  <em>/
zone_wan_dest_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_wan_input (1 references)
target prot opt source destination
input_wan_rule all -- anywhere anywhere /</em>  !fw3: Custom wan input rule chain  <em>/
ACCEPT udp -- anywhere anywhere udp dpt:bootpc /</em>  !fw3: Allow-DHCP-Renew  <em>/
ACCEPT icmp -- anywhere anywhere icmp echo-request /</em>  !fw3: Allow-Ping  <em>/
ACCEPT igmp -- anywhere anywhere /</em>  !fw3: Allow-IGMP  <em>/
ACCEPT all -- anywhere anywhere ctstate DNAT /</em>  !fw3: Accept port redirections  <em>/
zone_wan_src_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_wan_output (1 references)
target prot opt source destination
output_wan_rule all -- anywhere anywhere /</em>  !fw3: Custom wan output rule chain  <em>/
zone_wan_dest_ACCEPT all -- anywhere anywhere /</em>  !fw3  <em>/
Chain zone_wan_src_ACCEPT (1 references)
target prot opt source destination
ACCEPT all -- anywhere anywhere ctstate NEW,UNTRACKED /</em>  !fw3 */

Best regards
Byeon Seongsu

anon50098793 · October 28, 2019, 11:00am

One host is missing a route

Firewall settings... the WAN port is "facing the internet"... you will need to add some rules to allow what you want...

qussungtn · October 28, 2019, 11:02am

I checked the the traffic with "tcpdump".

It shows two problems when I ping these two devices from the each side.

raspberry pi(connected to WAN port) could send a ping request to controller(connected LAN port). But controller does not reply.
Controller(connected to LAN port) could not send a ping request to WAN port and raspberry.

it is not easy to understand the this issue.
Best regards
byeon seongsu

anon50098793 · October 28, 2019, 11:09am

you mentioned that already... it was helpful when I informed you what to check next.... like showing us the route output on the client devices.... might be worth checking out....

ulmwind · October 28, 2019, 11:10am

You should enable firewall on OpenWRT router, because it makes NAT.

qussungtn · October 28, 2019, 12:35pm

sorry. I cannot get your point.
Can I ask you two questions?

what is the relationship between firewall setting and NAT?
which firewall setting should I enalble?

best regards.
Byeon Seongsu

ulmwind · October 28, 2019, 1:28pm

NAT is performed by nat-table of iptables. If you disable firewall completely, I don't understand, who should forward packets. In default configuration with setting only WAN IP you should be able to ping from LAN to WAN. Can you do it?

qussungtn · October 28, 2019, 1:30pm

Dear wulfy23,

Thank you for your advice.

LAN port(192.168.0.254) is reachable from raspberry pie. But controller(192.168.0.252) is not reachable. Tcpdump shows that controller does not reply to the request from rasspberry

I fixed this issue by setting default gateway ip address as below. It means that there is no problem from WAN to LAN routing.

Capture.JPG872×266 26.5 KB

Capture2.JPG747×548 129 KB

WAN port(192.168.1.254) is not reachable from controller(192.168.0.252)."

As I mentioned above, routing from WAN to LAN is OK. But now the only one problem is that controller(192.168.0.252) does not send a ping request to neither WAN port(192.168.1.254). It could only ping to LAN port(192.168.0.254)
Is it true that " "All traffic from the outside into WAN (including ICMP/pings) is blocked by default." ?

qussungtn · October 28, 2019, 1:48pm

Dear ulmwind,

Thank you for your reply.
I am sorry for my lack of knowledge in networking.

First of all,
I can ping from WAN to LAN but not from LAN to WAN.
it means that from the Raspberry pi (connected to WAN), I can ping to the device connected to LAN port.

Secondly,

I did not disable all the firewalls. I set some options as "ACCEPT"
it is not possible to ping from the controller(connected to LAN port) to the WAN port(192.168.1.254). Even it does not send a ping request.

Best regards
Byeon seongsu.

ulmwind · October 28, 2019, 1:51pm

Sorry, can you ping controller from RPi?

qussungtn · October 28, 2019, 1:54pm

yes I can ping from RPi to controller.
But from the controller, I can ping only LAN port.

rj-45 · October 28, 2019, 1:55pm

how did you do that?

ulmwind · October 28, 2019, 1:56pm

Can you ping RPi from OpenWRT router?

qussungtn · October 28, 2019, 1:58pm

sorry.

This comment makes a confusion.

I disabled the firewall setting of my laptop(controller)
I did not disable all the firewalls of openWrt Router. I set some options which were "REJECT" as "ACCEPT"

Best regards
Byeon Seongsu

qussungtn · October 28, 2019, 2:22pm

yes I can ping both devices form Openwrt Router!
Problem is from controller(connected to LAN port) to WAN port. Controller does not send any ping request to WAN port. I checked it with tcpdump tool.

ulmwind · October 28, 2019, 2:26pm

It is very strange behavior. I'll write you some considerations.

Forwarding between zones is defined by config forwarding sections, so in default configuration only lan-to-wan forwarding is enabled.
Forwarding to wan in default configuration includes masquerading (option masq of wan interface).

So you have 'inverse' behavior to default configuration, I don't understand, why. Could you check and establish ssh-connection to controller from RPi?

qussungtn · October 28, 2019, 2:36pm

Thank you for your opinion.

I checked ssh-connection from RPi and the connection was established.

ssh root@192.168.0.254
RPi static ip address: 192.168.1.253
the forwarding rule is defined as below.
config forwarding
option src lan
option dest wan

It seems strage for me.

Best regards
Byeon Seongsu

ulmwind · October 28, 2019, 2:46pm

OK, maybe it is due to the fact, that you've enabled forwarding policy completely. My common recommendation is not to enable everything, but step-by-step from default configuration, checking changing in behavior. However the issue of not pinging from controller to RPi is very strange. My only guess is that it is due to enabling of IPv6, I've found your error message: Ping is a valuable tool for troubleshooting all sorts of network or Internet problems. A ping command sends packets of information to devices on a network or the Internet to find out if they are available for connection. In Windows 8, the error message "Ping: Transmit Failed. General Failure" appears if the computer doesn't have the proper Internet protocol selected. You may select TCP/IPv4 or TCP/IPv6 in the control panel of your computer.