Dropped incoming traffic

Hi,

I have a problem with with processing incomming packets on netgear r7800 23.05.4 flowoffloading=off

Here I am pinging the WAN GW:

root@OpenWrt-SW:~# ping -s 500 -c 100 10.0.0.1
PING 10.0.0.1 (10.0.0.1): 500 data bytes
508 bytes from 10.0.0.1: seq=0 ttl=64 time=0.471 ms
508 bytes from 10.0.0.1: seq=1 ttl=64 time=0.499 ms
508 bytes from 10.0.0.1: seq=2 ttl=64 time=0.497 ms
508 bytes from 10.0.0.1: seq=3 ttl=64 time=0.496 ms
508 bytes from 10.0.0.1: seq=4 ttl=64 time=0.496 ms
508 bytes from 10.0.0.1: seq=5 ttl=64 time=0.433 ms
508 bytes from 10.0.0.1: seq=6 ttl=64 time=0.421 ms
508 bytes from 10.0.0.1: seq=7 ttl=64 time=0.396 ms
508 bytes from 10.0.0.1: seq=11 ttl=64 time=0.500 ms
508 bytes from 10.0.0.1: seq=12 ttl=64 time=0.600 ms
^C
--- 10.0.0.1 ping statistics ---
13 packets transmitted, 10 packets received, 23% packet loss
round-trip min/avg/max = 0.396/0.480/0.600 ms

However, looking at tcpudmp on the same device, all replies seem to be returning (seq 7-10):

root@OpenWrt-SW:~# tcpdump -i any -n "icmp and host 10.0.0.1" | grep "length 508"
...
23:42:39.971840 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 6, length 508
23:42:39.972161 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 6, length 508
23:42:39.972161 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 6, length 508
23:42:40.972364 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 7, length 508
23:42:40.972666 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 7, length 508
23:42:40.972666 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 7, length 508
23:42:41.972687 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 8, length 508
23:42:41.973006 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 8, length 508
23:42:41.973006 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 8, length 508
23:42:42.973196 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 9, length 508
23:42:42.973498 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 9, length 508
23:42:42.973498 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 9, length 508
23:42:43.973946 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 10, length 508
23:42:43.974263 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 10, length 508
23:42:43.974263 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 10, length 508
23:42:44.974753 eth0.2 Out IP 10.0.0.186 > 10.0.0.1: ICMP echo request, id 8938, seq 11, length 508
23:42:44.975048 eth0  P   IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 11, length 508
23:42:44.975048 eth0.2 In  IP 10.0.0.1 > 10.0.0.186: ICMP echo reply, id 8938, seq 11, length 508

I have noticed it after update from 23.05.3 -> 23.05.4 however downgrading does not help, problem remains. CPU and mem usage seems normal.
This is also affecting other in/out and forwarded traffic.
Firewall is static/normal + banIP.

Would you have some ideas how to further debug this issue?

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/firewall
2 Likes

let's also see the output of ifstatus wan | grep address.

If the output of that contains anything that starts with 192.168, 172.16 - 172.31, or 10., you don't need to redact it. If it's something different, though, show us only the first to octets (in bold: aaa.bbb.ccc.ddd)

1 Like

Hi,

Thanks for the extra command suggestion.
This setup has some complications - multiwan, policy routing (for wan) by marking packets and conntrack.

root@OpenWrt-SW:~# ubus call system board
{
        "kernel": "5.15.162",
        "hostname": "OpenWrt-SW",
        "system": "ARMv7 Processor rev 0 (v7l)",
        "model": "Netgear Nighthawk X4S R7800",
        "board_name": "netgear,r7800",
        "rootfs_type": "squashfs",
        "release": {
                "distribution": "OpenWrt",
                "version": "23.05.4",
                "revision": "r24012-d8dd03c46f",
                "target": "ipq806x/generic",
                "description": "OpenWrt 23.05.4 r24012-d8dd03c46f"
        }
}

The 10.a.b.c are already redacted public IPes.
Wan1 and Wan2 are static IPes. Wan-3g is a dhcp.
/etc/config has no default routes for interfaces.
They are configured in separate routing tables + default table. There is default route in main table.

root@OpenWrt-SW:~# ifstatus wan | grep address
                "addresses",
        "ipv4-address": [
                        "address": "10.0.0.186",
        "ipv6-address": [
                "ipv4-address": [
                "ipv6-address": [

Perhaps it would be easier to have a look at the following output:

root@OpenWrt-SW:~# ip rule
0:      from all lookup local
1000:   from all lookup main
2000:   from all fwmark 0xe lookup wan-isp1
3000:   from all fwmark 0xf lookup wan-isp2
4000:   from all fwmark 0x10 lookup wan-3g
32766:  from all lookup main
32767:  from all lookup default
root@OpenWrt-SW:~# ip r show table local
local 10.0.0.186 dev eth0.2 proto kernel scope host src 10.0.0.186
...

Incoming traffic first goes to local table which has the local IP.
So routing wise everything should be fine. Would you agree with that?

This drops are happening not so often - only sometimes incoming packets are not processed.
As for the firewall - it is static (apart from banIP). Marking and conntrack are for outgoing traffic. Based on marking routing is done (just like showed in "ip rule").