Ping replies from modem to br-lan

Hello there!

I am stuck at configuring vpn-policy-routing. I have a BTHH5A running OpenWrt 19.07.4 r11208-ce6496d796. My ISP is Virgin Media UK. I am using their superhub as a modem.

Coming to my situation ....

My mission : Access the modem having ip address 192.168.100.1 ( given by ISP ) over https. The modem is listening on port 443.

What I did : I followed the instructions given here : https://openwrt.org/docs/guide-user/network/wan/access.modem.through.nat#web_interface_instructions

My situation : I have a wireguard VPN with an interface WG0IF. To route the default traffic over WG-VPN, I installed package vpn-policy-routing. The WG0IF and modem interfaces share firewall zone with WAN.

When the VPR servie is running, I can't ping from my laptop which is a br-lan's client. The url https://192.168.100.1:443 gives a timeout error. All other traffic to internet is routed through WG0IF, as intended.

When I stop the PBR service, my laptop can ping the modem. The url https://192.168.100.1:443 responds with a login page. All other traffic to internet is routed through WAN.

My question : How do I make modem accessible from br-lan client while PBR service is still running?

The relevent details are :


PBR details :

config policy
        option name 'modem'
        option proto 'all'
        option src_addr '192.168.0.0/24'
        option dest_addr '192.168.100.1'
        option interface 'wan'

config policy
        option interface 'wan'
        option name 'br-lan-internal'
        option src_addr '192.168.0.0/24'
        option dest_addr '192.168.0.0/24'
        option proto 'all'

config policy
        option interface 'WG0IF'
        option name 'br-lan'
        option src_addr '192.168.0.0/16'

config vpn-policy-routing 'config'
        option verbosity '2'
        option src_ipset '0'
        option dest_ipset 'dnsmasq.ipset'
        option ipv6_enabled '0'
        option boot_timeout '30'
        option iptables_rule_option 'append'
        option iprule_enabled '0'
        option webui_sorting '1'
        list webui_supported_protocol 'tcp'
        list webui_supported_protocol 'udp'
        list webui_supported_protocol 'tcp udp'
        list webui_supported_protocol 'icmp'
        list webui_supported_protocol 'all'
        option webui_enable_column '1'
        option webui_protocol_column '1'
        option webui_chain_column '1'
        option strict_enforcement '0'
        option resolver_ipset 'none'
        option webui_show_ignore_target '1'
        option enabled '1'

----------------------------

Firewall defaults :

~# uci show firewall| grep @defaults

firewall.@defaults[0]=defaults
firewall.@defaults[0].input='ACCEPT'
firewall.@defaults[0].output='ACCEPT'
firewall.@defaults[0].forward='REJECT'
firewall.@defaults[0].flow_offloading='1'
firewall.@defaults[0].flow_offloading_hw='1'
firewall.@defaults[0].synflood_protect='1'

----------------------------

Firewall zone details :

~# uci show firewall| grep zone

firewall.@zone[0]=zone
firewall.@zone[0].name='lan'
firewall.@zone[0].input='ACCEPT'
firewall.@zone[0].output='ACCEPT'
firewall.@zone[0].forward='ACCEPT'
firewall.@zone[0].network='lan'
firewall.@zone[1]=zone
firewall.@zone[1].name='wan'
firewall.@zone[1].output='ACCEPT'
firewall.@zone[1].mtu_fix='1'
firewall.@zone[1].masq='1'
firewall.@zone[1].network='wan wan6 WG0IF modem'
firewall.@zone[1].input='REJECT'
firewall.@zone[1].forward='REJECT'

----------------------------

Firewall forwarding details :

~# uci show firewall| grep forwarding

firewall.@forwarding[0]=forwarding
firewall.@forwarding[0].src='lan'
firewall.@forwarding[0].dest='wan'

----------------------------

To investigate further, I ran tcpdump.

When PBR servie is down, I can see the ping request and reply messages in br-lan :point_down: :


~# tcpdump -vv -i br-lan | grep 192.168.100.1
tcpdump: listening on br-lan, link-type EN10MB (Ethernet), capture size 262144 bytes
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 98, length 40
    192.168.100.1 > my-laptop: ICMP echo reply, id 1, seq 98, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 99, length 40
    192.168.100.1 > my-laptop: ICMP echo reply, id 1, seq 99, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 100, length 40
    192.168.100.1 > my-laptop: ICMP echo reply, id 1, seq 100, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 101, length 40
    192.168.100.1 > my-laptop: ICMP echo reply, id 1, seq 101, length 40

When PBR servie is running, I see the ping requests but not any reply message in br-lan :point_down: :


~# tcpdump -vv -i br-lan | grep 192.168.100.1
tcpdump: listening on br-lan, link-type EN10MB (Ethernet), capture size 262144 bytes
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 90, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 91, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 92, length 40
    my-laptop > 192.168.100.1: ICMP echo request, id 1, seq 93, length 40

Irrespective of PBR status, ping requests and replies are always visible on WAN interface :point_down: :

~# tcpdump -vv -i eth0.2 | grep 192.168.100.1
tcpdump: listening on eth0.2, link-type EN10MB (Ethernet), capture size 262144 bytes
    cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net > 192.168.100.1: ICMP echo request, id 1, seq 94, length 40
    192.168.100.1 > cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net: ICMP echo reply, id 1, seq 94, length 40
    cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net > 192.168.100.1: ICMP echo request, id 1, seq 95, length 40
    192.168.100.1 > cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net: ICMP echo reply, id 1, seq 95, length 40
    cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net > 192.168.100.1: ICMP echo request, id 1, seq 96, length 40
    192.168.100.1 > cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net: ICMP echo reply, id 1, seq 96, length 40
    cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net > 192.168.100.1: ICMP echo request, id 1, seq 97, length 40
    192.168.100.1 > cpc93712-lon16-4-0-cust31.4-8.cable.virginm.net: ICMP echo reply, id 1, seq 97, length 40

I can say PBR service is stopping the ping replies from WAN to enter br-lan. How do I allow them? Once ping is sorted out, I will check with https over port 443.

Thank you!

-Gamma

What is the output of iptables-save -c -t mangle ?

not sure exactly what is going on with your setup

but for source policy routing you can double check the device section for the "Reverse path filter" option

I think it needs to be disabled for policy routing to work

Hi @trendy ,

I've pasted the output below :

~# iptables-save -c -t mangle
# Generated by iptables-save v1.8.3 on Sat Jan  8 10:17:03 2022
*mangle
:PREROUTING ACCEPT [1123068:1049953991]
:INPUT ACCEPT [1087356:1043128595]
:FORWARD ACCEPT [23429:3298915]
:OUTPUT ACCEPT [873326:427997586]
:POSTROUTING ACCEPT [896358:431275768]
:VPR_FORWARD - [0:0]
:VPR_INPUT - [0:0]
:VPR_OUTPUT - [0:0]
:VPR_PREROUTING - [0:0]
[1123121:1049970262] -A PREROUTING -m mark --mark 0x0/0xff0000 -j VPR_PREROUTING
[1082853:1042704491] -A INPUT -m mark --mark 0x0/0xff0000 -j VPR_INPUT
[8502:515235] -A FORWARD -o WG0IF -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[8654:488724] -A FORWARD -i WG0IF -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[164:9060] -A FORWARD -o eth0.2 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[185:10052] -A FORWARD -i eth0.2 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone wan MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[0:0] -A FORWARD -o tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone tun0fwz MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[0:0] -A FORWARD -i tun0 -p tcp -m tcp --tcp-flags SYN,RST SYN -m comment --comment "!fw3: Zone tun0fwz MTU fixing" -j TCPMSS --clamp-mss-to-pmtu
[9918:1745015] -A FORWARD -m mark --mark 0x0/0xff0000 -j VPR_FORWARD
[873368:428000311] -A OUTPUT -m mark --mark 0x0/0xff0000 -j VPR_OUTPUT
[0:0] -A VPR_FORWARD -m set --match-set WG0IF dst -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_FORWARD -m set --match-set tun0if dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_FORWARD -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set WG0IF dst -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set tun0if dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_INPUT -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set WG0IF dst -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set tun0if dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_OUTPUT -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000
[30339:5513900] -A VPR_PREROUTING -s 192.168.0.0/24 -m comment --comment br-lan-2-wg -j MARK --set-xmark 0x30000/0xff0000
[4546:433483] -A VPR_PREROUTING -s 192.168.0.0/24 -d 192.168.0.0/24 -m comment --comment br-lan-internal -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_PREROUTING -s 192.168.0.0/24 -d 192.168.100.1/32 -m comment --comment br-lan-2-modem -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set WG0IF dst -j MARK --set-xmark 0x30000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set tun0if dst -j MARK --set-xmark 0x20000/0xff0000
[0:0] -A VPR_PREROUTING -m set --match-set wan dst -j MARK --set-xmark 0x10000/0xff0000
COMMIT
# Completed on Sat Jan  8 10:17:03 2022

Thank you.
-Gamma

Hi @wilsonyan ,

Sorry, I couldn't find any such option in vpn-policy-routing's conf file. Should I create one and explicitely disable it?

A quick googling suggested I run :

cat /proc/sys/net/ipv4/conf/*/rp_filter

They are all set to 0.

Thank you,
-Gamma

The rule you have for accessing the modem is below the rule for directing everything from lan to WG. You need to reverse the order.

[30339:5513900] -A VPR_PREROUTING -s 192.168.0.0/24 -m comment --comment br-lan-2-wg -j MARK --set-xmark 0x30000/0xff0000
[4546:433483] -A VPR_PREROUTING -s 192.168.0.0/24 -d 192.168.0.0/24 -m comment --comment br-lan-internal -j MARK --set-xmark 0x10000/0xff0000
[0:0] -A VPR_PREROUTING -s 192.168.0.0/24 -d 192.168.100.1/32 -m comment --comment br-lan-2-modem -j MARK --set-xmark 0x10000/0xff0000

The rule br-lan-internal is not necessary as this kind of traffic won't go through the router. Only packets destined to the OpenWrt will hit this rule.

Hi @trendy ,

Let me take out the br-lan-internal rule. That's coming from PBR config. How do I reverse positions of wg and modem rules? Can I edit the iptables directly?

Thank you.
-Gamma

You can move the rules in the Luci page, using the Up/Down buttons on the right side

Got it. Would try it out. So, no need to edit the iptables. Good to hear that.:slight_smile:
Thanks!
-Gamma

Hi @trendy,

I removed the br-lan-internal rule and that made the things work as intended. Perhaps this rule was preventing ICMP reply traffic from entering the br-lan.

I ensured that the PBR rule priority shows br-lan-2-modem above br-lan-2-wg rule. Still iptables show them other way round. Don't know why.

Despite of the contradiction, I can access the modem over both ping and https while PBR is running.

Misson accomplished. :slight_smile: The only problem now left is rebooting does not bring up PBR service in a clean way. I have to restart it. But that is another issue.

Thankyou for your help.

-Gamma

Edit : Grammer and further explanation.

1 Like

If your problem is solved, please consider marking this topic as [Solved]. See How to mark a topic as [Solved] for a short how-to.

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.