Ping from a specific interface not working

Little bit background

I am using GLinet AR-150. It has 3 interfaces: eth0 (single WAN port), eth1 (single LAN port), and wlan0 (WIFI). eth1 and wlan0 are bridged together (br-lan).

I recently got 2 connections from 2 different ISPs and thought to remove eth1 from the bridge and use it as a 2nd WAN port.

Both of my ISPs provided static IPs for router configuration.

ISP-1 (172.22.36.151/25) (Planned about connecting it to WAN-eth0)
ISP-2 (172.30.138.151/24) (Planned about connecting it to WAN2-eth1)

I used the following network config /etc/config/network

config interface 'loopback'
    option ifname 'lo'
    option proto 'static'
    option ipaddr '127.0.0.1'
    option netmask '255.0.0.0'

config globals 'globals'
    option ula_prefix 'fd22:bbe3:e6d8::/48'

config interface 'wan'
    option ifname 'eth0'
    option proto 'static'
    option ipaddr '172.22.36.151'
    option netmask '255.255.255.128'
    option gateway '172.22.36.129'
    option dns '8.8.8.8'
    option metric '10'

config interface 'wan2'
    option proto 'static'
    option ifname 'eth1'
    option ipaddr '172.30.138.151'
    option netmask '255.255.255.0'
    option gateway '172.30.138.1'
    option dns '8.8.8.8'
    option macaddr 'E4:95:6E:42:97:33'
    option metric '20'

config interface 'wan6'
    option ifname 'eth0'
    option proto 'dhcpv6'

config interface 'lan'
#REMOVED THE ETH1 WLAN0 BRIDGE. I WILL USE THIS LOGICAL INTERFACE `LAN` IN /ETC/CONFIG/WIRELESS
    option proto 'static'
    option netmask '255.255.255.0'
    option ip6assign '60'
    option hostname 'GL-AR150-732'
    option ipaddr '192.168.8.1'

image

The outputs of ip a show and ip route show are attached at the end of this post.

Now, I connected the ISP cables to the respective pre-planned ports.

IMG_20190126_172226

Now, I am specifying the interface to use for ping using:

ping -I eth0 8.8.8.8
ping -I eth1 8.8.8.8

Expected outcome

In the above commands, the ping should be sent via eth0 and eth1 respectively. The expected outcome is also mentioned in the mwan3 documentation.

image

The problem I am facing

Both the ping commands are being sent from eth1

Most probably because eth1 gateway has a lower metric.

If I use lower metric for eth0 gateway, then all pings are being sent from eth0 regardless of what interface I specify.

This behavior is similar to IP aliasing on a single interface. But I am using 2 completely different interfaces and if I specify the interface to use, then the default gateway metric should not matter. (see the expected outcome screenshot, they have 2 gateway with different metrics) The ping should be sent from my specified interface.

Please help.

Output of "ip route show"

default via 172.30.138.1 dev eth1 src 172.30.138.151 metric 10 
default via 172.22.36.129 dev eth0 src 172.22.36.151 metric 20 
172.22.36.128/25 dev eth0 proto static scope link metric 10 
172.30.138.0/24 dev eth1 proto static scope link metric 20 
192.168.8.0/24 dev wlan0 proto kernel scope link src 192.168.8.1

Output of "ip a show"

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
   valid_lft forever preferred_lft forever
inet6 ::1/128 scope host 
   valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether e4:95:6e:42:97:32 brd ff:ff:ff:ff:ff:ff
inet 172.22.36.151/25 brd 172.22.36.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9732/64 scope link 
   valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
link/ether e4:95:6e:42:97:33 brd ff:ff:ff:ff:ff:ff
inet 172.30.138.151/24 brd 172.30.138.255 scope global eth1
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9733/64 scope link dadfailed tentative 
   valid_lft forever preferred_lft forever
4: teql0: <NOARP> mtu 1500 qdisc noop state DOWN group default qlen 100
link/void 
9: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether e4:95:6e:42:97:32 brd ff:ff:ff:ff:ff:ff
inet 192.168.8.1/24 brd 192.168.8.255 scope global wlan0
   valid_lft forever preferred_lft forever
inet6 fd22:bbe3:e6d8::1/60 scope global noprefixroute 
   valid_lft forever preferred_lft forever
inet6 fe80::e695:6eff:fe42:9732/64 scope link 
   valid_lft forever preferred_lft forever

Either use mwan3, or perform the routing properly:

1 Like

I tried to use mwan3 but ran into some problem (both interface status was showing 'online' even when I disconnected one cable).

Then I started to read the mwan3 documentation carefully. In that documentation, they suggested that before using mwan3, the user need to check if the pings from both interfaces are working.

image

I did exactly what they said and my routing table is also similar. But still the ping is being sent from only one specific interface with lower gateway metric irrespective of which interface I specify. This is the problem I am facing and I want to understand why.

Maybe my original post is too long but I thought to leave as many details as possible.

ip route list table all
ip rule list
iptables-save

It is not clear to me how did you verify that exactly (tcpdump or something like that?). Also in the mwan3 documentation it is only mentioned that the test is successful if the ping is successful.

  • Also...how did you specify the Interface?
  • Did you perform this ping test from the router?
  • Did you do this prior to installing mwan3?

ping 8.8.8.8 -I <IP_of_each_Interface>

I would surmise that test should have been done prior to installing the package.

All you needed to do to enter this state was:

  • remove the bridge from LAN
  • This should have placed wlan0 in its own firewall LAN zone
  • Create a WAN2

It seems you've done that.

  1. ping with -I switch. Tried specifying both interface name and interface IP address.
  2. yes
  3. yes

That is exactly what I did. Everything is working fine. I can connect to my wifi and connect to the internet. The only problem is that the ping is not being sent from my specified interface.

tcpdump is a quite heavy package, I think. Thus I didn't install that. I verified it using 2 methods.

  1. I know the ping response time from a specific ISP. ISP-1 takes around 50 ms and ISP-2 takes around 80 ms to receive a ping response from 8.8.8.8 Thus, I can look at the output roundtrip time and easily understand which ISP network has been used.
  2. I used -i (interval) switch of the ping utility and sent a ping every 0.1 seconds. Luci web GUI refreshes every 3 seconds or so. If only one interface is being used and the other one is not, then it will show significant packet count difference since the last refresh, in the tx packet statistics of the interfaces.

And at the mwan3 documentation, you can see that the packets were actually sent from the 2 different networks because the round trip time is different in case of wan1 and wan2.

Please wait for some time. I currently don't have access to the router. I will post the outputs soon.

I am not convinced by your method. There is no solid result. Also pinging with source IP will always return result. Stick to the interface names.

root@xeli:~# ping -I 10.14.149.65 -c 1 1.1.1.1
PING 1.1.1.1 (1.1.1.1) from 10.14.149.65: 56 data bytes
64 bytes from 1.1.1.1: seq=0 ttl=57 time=7.010 ms

--- 1.1.1.1 ping statistics ---
1 packets transmitted, 1 packets received, 0% packet loss
round-trip min/avg/max = 7.010/7.010/7.010 ms

Either install the tcpdump, or you could ping the gateway of each interface instead of 8.8.8.8

This may be the reason. In windows, we need to specify interface IP to send the packet through a specific interface. But on Linux, we need to specify the interface name. Otherwise, the packet will be sent via the default route with the lowest metric. I can't remember whether I used interface IP by mistake or it was a misconfigured routing table because I did a full reset on my router and started from a fresh install after I failed to achieve the desired result. And then it worked perfectly as indicated by the below screenshot (left ISP1 and right ISP2). Thanks to everyone who helped.

I did face some challenges while trying to reach my 2nd ISP IP from the internet. I specified both ISP gateway as default route in the routing table but ISP1 had a lower metric. So, even if the ping echo request came on eth1 , the echo reply was sent out from eth0 . On windows, it is not a problem because the response is sent out automatically from the original interface where it is received. But here I resolved it by deploying policy-based routing using ip rule

echo "1 isp2" >> /etc/iproute2/rt_tables
ip route add default via 172.30.138.1 dev eth1 table isp2
ip rule add from 172.30.138.151/32 table isp2

Now the configuration is complete. Thus if anyone wants to use a similar setup on AR150, they can just use the network configuration posted on my original question. It is perfectly fine.

I finally installed tcpdump but facing a problem.
tcpdump should by default show both incoming packets and outgoing packets on the specified interface. It also has a switch -Q in|out|inout to specify the packet capture direction. In my Ubuntu computer, it is working as expected but on OpenWRT, if I specify a specific interface -i eth0|eth1, it is only showing incoming packets but if I specify -i any only then all packets are showing.

It can be seen from the following picture. with or without specifying -Q, only the ICMP echo reply is showing but with interface any, both request and reply is showing. Any help regarding this will be appreciated. Thank you :slight_smile:

I was not able to reproduce your issue with tcpdump.

root@fagri:~# tcpdump -i pppoe-wan -tttt -e -n icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on pppoe-wan, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
2019-01-31 17:52:44.549078 Out ethertype IPv4 (0x0800), length 100: 89.210.XXX.XXX > XXX.XXX.80.1: ICMP echo request, id 53874, seq 136, length 64
2019-01-31 17:52:44.552154 Out ethertype IPv4 (0x0800), length 100: 89.210.XXX.XXX > 8.8.8.8: ICMP echo request, id 54130, seq 136, length 64
2019-01-31 17:52:44.590575  In ethertype IPv4 (0x0800), length 100: XXX.XXX.80.1 > 89.210.XXX.XXX: ICMP echo reply, id 53874, seq 136, length 64
2019-01-31 17:52:44.590581  In ethertype IPv4 (0x0800), length 100: 8.8.8.8 > 89.210.XXX.XXX: ICMP echo reply, id 54130, seq 136, length 64

However, before filling a bug, make sure that packets don't leave from another interface. Open a lot of tcpdump instances, each for a different interface and see where the echo requests will show up.

I noticed that you are using tcpdump on a PPPoE interface, and thus, tcpdump is using Linux cooked-mode capture (SLL). This mode is used when you use "tcpdump -i any" or other interfaces like PPP where the ethernet header can't be retrieved reliably (More about this is discussed on the above link). You can see on my previous screenshot that I was getting both echo request and echo reply at the last time when I was in SLL /cooked mode.

Is it kindly possible for you to run tcpdump on router LAN interface and verify? (because then tcpdump won't use SLL link type)

Worked as well :slight_smile:

root@OpenWrt:/# tcpdump -i eth1 -tttt -e -n icmp
[ 1240.443277] device eth1 entered promiscuous mode
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), capture size 262144 bytes
2019-01-31 20:37:41.541479 c4:93:00:07:2e:6c > c4:93:00:00:1b:0d, ethertype IPv4 (0x0800), length 98: 10.14.149.65 > 10.14.149.76: ICMP echo request, id 10272, seq 0, length 64
2019-01-31 20:37:41.541746 c4:93:00:00:1b:0d > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.76 > 10.14.149.65: ICMP echo reply, id 10272, seq 0, length 64
2019-01-31 20:37:42.541785 c4:93:00:07:2e:6c > c4:93:00:00:1b:0d, ethertype IPv4 (0x0800), length 98: 10.14.149.65 > 10.14.149.76: ICMP echo request, id 10272, seq 1, length 64
2019-01-31 20:37:42.542003 c4:93:00:00:1b:0d > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.76 > 10.14.149.65: ICMP echo reply, id 10272, seq 1, length 64
2019-01-31 20:37:43.542072 c4:93:00:07:2e:6c > c4:93:00:00:1b:0d, ethertype IPv4 (0x0800), length 98: 10.14.149.65 > 10.14.149.76: ICMP echo request, id 10272, seq 2, length 64
2019-01-31 20:37:43.542281 c4:93:00:00:1b:0d > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.76 > 10.14.149.65: ICMP echo reply, id 10272, seq 2, length 64
^C
6 packets captured
6 packets rec[ 1261.523922] device eth1 left promiscuous mode
eived by filter
0 packets dropped by kernel

This one is performed on a brand new 18.06.2

root@xeli:~# tcpdump -i eth0 -tttt -e -n icmp
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
2019-01-31 21:40:09.737310 c4:93:00:07:2e:6c > b8:27:eb:72:35:f3, ethertype IPv4 (0x0800), length 98: 10.14.149.225 > 10.14.149.119: ICMP echo request, id 12684, seq 9, length 64
2019-01-31 21:40:09.737933 b8:27:eb:72:35:f3 > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.119 > 10.14.149.225: ICMP echo reply, id 12684, seq 9, length 64
2019-01-31 21:40:10.710971 c4:93:00:07:2e:6c > b8:27:eb:72:35:f3, ethertype IPv4 (0x0800), length 98: 10.14.149.225 > 10.14.149.119: ICMP echo request, id 12684, seq 10, length 64
2019-01-31 21:40:10.711587 b8:27:eb:72:35:f3 > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.119 > 10.14.149.225: ICMP echo reply, id 12684, seq 10, length 64
2019-01-31 21:40:10.844015 c4:93:00:07:2e:6c > b8:27:eb:72:35:f3, ethertype IPv4 (0x0800), length 98: 10.14.149.225 > 10.14.149.119: ICMP echo request, id 12692, seq 1, length 64
2019-01-31 21:40:10.844652 b8:27:eb:72:35:f3 > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.119 > 10.14.149.225: ICMP echo reply, id 12692, seq 1, length 64
2019-01-31 21:40:11.886068 c4:93:00:07:2e:6c > b8:27:eb:72:35:f3, ethertype IPv4 (0x0800), length 98: 10.14.149.225 > 10.14.149.119: ICMP echo request, id 12692, seq 2, length 64
2019-01-31 21:40:11.886698 b8:27:eb:72:35:f3 > c4:93:00:07:2e:6c, ethertype IPv4 (0x0800), length 98: 10.14.149.119 > 10.14.149.225: ICMP echo reply, id 12692, seq 2, length 64
^C
8 packets captured
13 packets received by filter
0 packets dropped by kernel

This one on a 18.06.1

Sorry for bothering you :sweat_smile:. Maybe I am missing something. I will let you know if I solve this later. :slight_smile:

No problem, good luck with it!

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.