How can I monitor arp traffic on the WAN?

Hi,

I have a network connected to CGNAT and I’m trying to figure out why I am getting moments where traffic stops being router ?

I suspect on the WAN side I’m getting some ARP issues, so I’d like to run some software to make sense of all the arp messages and monitor the status of the arp table and maybe figure out why things occasionally get misconfigured ?

I’ve found arp-scan and arp-whisper but I can’t figure out how to use them. arp-whisper needs a config file to work ?

And all I could search was for package with “arp” in the package name. Are there better utilities to monitor ARP and makes sense of what is going on ?

Install ip-full and use IP monitor. You can also verify the neighbor table with IP neighbor.

If i undesrtand you concern try this:
tcpdump -i wan arp

With that I get all arp traffic, not just mine but requests of other ISP customers with that command.

How do you know you jave ARP issues without monitoring it?
You can use arptables-nft to log count filter arp traffic

Hi,

I am able to monitor arp traffic with the following commands

apk add ip-full
ip monitor neigh dev wan

with timestamp

apk add ip-full moreutils
ip monitor neigh dev wan | ts

with timestamp including milliseconds

apk add ip-full moreutils coreutils-stdbuf
IP_FORCE_COLOR=1 stdbuf -oL ip monitor neigh dev wan | perl -MTime::HiRes -MPOSIX=strftime -ne '($s,$us)=Time::HiRes::gettim
eofday(); printf "\e[90m%s.%03d\e[0m %s", strftime("%Y-%m-%d %H:%M:%S", localtime($s)), $us/1000, $_'

Watching the arp traffic

tcpdump -i wan arp

or with more details

tcpdump -i wan -nne -vvv arp

thanks !

And here is what that generally looks like when things are working on my end.
This is a CGNAT setup

34:ef:b0:0b:ba:ab 100.66.66.1 is my CGNAT gateway
01:02:be:ef:ba:af 100.66.66.11 is my CGNAT host ip
66.66.66.141 would be my external IP address
66.66.66.109 66.66.66.110, I don't know what those are
(all addresses changed to protect the guilty)

2026-01-29 12:49:34.090 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:49:39.530 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:49:39.531 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:49:55.210 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:50:00.410 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:50:00.411 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:50:16.330 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:50:21.450 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:50:21.451 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:50:50.250 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:50:55.290 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:50:55.292 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:51:24.170 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:51:29.770 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:51:29.771 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:51:58.730 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:52:03.770 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:52:03.771 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:52:32.650 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:52:37.690 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:52:37.692 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:53:06.570 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:53:11.850 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:53:11.851 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:53:40.490 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:53:45.530 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:53:45.531 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE
2026-01-29 12:54:14.410 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab STALE
2026-01-29 12:54:19.930 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab PROBE
2026-01-29 12:54:19.931 100.66.66.1 lladdr 34:ef:b0:0b:ba:ab REACHABLE

12:50:00.409748 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:50:00.410621 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:00.410623 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:17.126900 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:18.176580 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:19.200457 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:21.449807 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:50:21.450930 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:21.450932 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:21.609631 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:22.656873 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:23.680420 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:25.540735 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:26.560440 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:27.584725 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:33.998976 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:35.008287 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:36.032537 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:40.020902 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:41.024531 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:42.048421 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:45.566672 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:46.592530 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:47.616470 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:52.421532 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:53.440360 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:54.464869 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:55.289749 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:50:55.291605 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:55.291607 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:50:55.784049 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:56.832406 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:50:57.856367 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:10.602877 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:11.616420 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:12.640339 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:21.726346 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:22.752643 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:23.776347 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:25.347707 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:26.400606 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:27.424374 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:29.769739 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:51:29.771227 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:51:29.771230 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:51:32.123515 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:33.185470 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:34.208456 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:38.948704 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:39.968401 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:40.992579 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:51:58.962210 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:00.000718 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:01.024498 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:03.769740 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:52:03.770654 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:52:03.770656 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:52:11.064419 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:12.096543 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:13.120448 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:18.989803 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:20.032560 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:21.056401 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:37.689744 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:52:37.691512 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:52:37.691515 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:52:43.435627 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:44.480655 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:52:45.504442 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:00.835052 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:01.888831 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:02.912459 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:11.849762 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:53:11.851192 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:53:11.851194 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:53:12.927283 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:13.984579 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:15.008448 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:19.772982 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:20.832705 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:21.856513 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:45.529776 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:53:45.530920 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:53:45.530923 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:53:55.537457 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:56.544392 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:53:57.568579 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:54:05.429895 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:54:06.464630 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:54:07.488467 34:ef:b0:0b:ba:ab > ff:ff:ff:ff:ff:ff, ethertype 802.1Q (0x8100), length 60: vlan 50, p 0, ethertype ARP (0x0806), Ethernet (len 6), IPv4 (len 4), Request who-has 66.66.66.110 tell 66.66.66.109, length 42
12:54:19.929755 01:02:be:ef:ba:af > 34:ef:b0:0b:ba:ab, ethertype ARP (0x0806), length 42: Ethernet (len 6), IPv4 (len 4), Request who-has 100.66.66.1 tell 100.66.66.11, length 28
12:54:19.930716 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46
12:54:19.930718 34:ef:b0:0b:ba:ab > 01:02:be:ef:ba:af, ethertype ARP (0x0806), length 60: Ethernet (len 6), IPv4 (len 4), Reply 100.66.66.1 is-at 34:ef:b0:0b:ba:ab, length 46

No need:

# ip -ts monitor neigh
# man 8 ip
...
       -ts, -tshort
              Like -timestamp, but use shorter format.
...
       -t, -timestamp
              display current time when using monitor option.

Sry, does not compute. What does it do, and why do you need it? What are you looking for?

Both belong to the same /30:

$ ipcalc 66.66.66.109/30
Address:   66.66.66.109         01000010.01000010.01000010.011011 01
Netmask:   255.255.255.252 = 30 11111111.11111111.11111111.111111 00
Wildcard:  0.0.0.3              00000000.00000000.00000000.000000 11
=>
Network:   66.66.66.108/30      01000010.01000010.01000010.011011 00
HostMin:   66.66.66.109         01000010.01000010.01000010.011011 01
HostMax:   66.66.66.110         01000010.01000010.01000010.011011 10
Broadcast: 66.66.66.111         01000010.01000010.01000010.011011 11
Hosts/Net: 2

ARP requests are only made if you share a common layer 2 and layer 3 network; are your netmasks correct everywhere?

What does your route table say?

ip route get <dest>

The perl line, I wanted to add millisecond timestamps like the tcpdump lines but also keep the colours intact. With this the color have become gray and white but the time do have millisecond timestamp that match tcpdump.

I will try -ts on ip, that would be most convenient

As for 66.66.66.109 66.66.66.110
I don't know why I see them, maybe neighbouring hosts

My CGNAT address in 100.66.66.11 and netmark is 255.255.240.0
My host address is 66.66.66.141/32

Strangely I never see any ARP traffic concerning 66.66.66.141

In my router,
I can't see any mentions of 66.66.66.141 in ifconfig,
it's not in "route" either

root@router:~# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         10.66.66.1     0.0.0.0         UG    0      0        0 wan
10.0.0.0        *               255.255.0.0     U     0      0        0 br-lan
10.66.66.0     *               255.255.240.0   U     0      0        0 wan
root@router:~#

only here

root@router:~# uci show network.public_loopback
network.public_loopback=interface
network.public_loopback.device='lo'
network.public_loopback.ipaddr='66.66.66.141'
network.public_loopback.proto='static'
network.public_loopback.netmask='255.255.255.255'
root@router:~#

and here

firewall.@nat[0]=nat
firewall.@nat[0].name='public_snat'
firewall.@nat[0].proto='all'
firewall.@nat[0].src='wan'
firewall.@nat[0].target='SNAT'
firewall.@nat[0].snat_ip='66.66.66.141'

here is the route for 8.8.8.8

root@router:~# ip route get 8.8.8.8
8.8.8.8 via 100.66.66.1 dev wan src 100.66.66.11 uid 0
    cache
root@router:~#

And to be clear, this is working like 99% of the time and very well too.

Yes Sure. I do not doubt. We still can't exclude issues on the provider side. Most ISP networks are build horrible cheap. Sadly.

Regarding route and ifconfig...

Could you do me a personal favor and start using ip route and ip addr? ip offers so much more then all depricated legacy tools. Since over 15 years by now... Kthxbye.

I still prefer the presentation of ifconfig but here you go
It did make the loopback address appear however

root@router:~# ip route
default via 100.66.66.1 dev wan proto static
10.0.0.0/16 dev br-lan proto kernel scope link src 10.0.0.1
100.66.66.0/20 dev wan proto kernel scope link src 100.66.66.11
root@router:~# ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet 66.66.66.141/32 brd 255.255.255.255 scope global lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host proto kernel_lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 qdisc mq state UP group default qlen 1000
    link/ether ba:30:8b:15:a4:66 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::b830:8bff:fe15:a466/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
3: wan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 01:02:be:ef:ba:af brd ff:ff:ff:ff:ff:ff
    inet 100.66.66.11/20 brd 100.87.47.255 scope global wan
       valid_lft forever preferred_lft forever
    inet6 fe80::b20f:beff:feef:6769/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
4: lan1@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
    link/ether 34:98:b5:0d:97:09 brd ff:ff:ff:ff:ff:ff
5: lan2@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN group default qlen 1000
    link/ether 34:98:b5:0d:97:09 brd ff:ff:ff:ff:ff:ff
6: lan3@eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue master br-lan state LOWERLAYERDOWN group default qlen 1000
    link/ether 34:98:b5:0d:97:09 brd ff:ff:ff:ff:ff:ff
7: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 34:98:b5:0d:97:09 brd ff:ff:ff:ff:ff:ff
    inet 10.0.0.1/16 brd 10.0.255.255 scope global br-lan
       valid_lft forever preferred_lft forever
    inet6 fdae:a05:e325::1/60 scope global noprefixroute
       valid_lft forever preferred_lft forever
    inet6 fe80::3698:b5ff:fe0d:9709/64 scope link proto kernel_ll
       valid_lft forever preferred_lft forever
8: phy0-ap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
    link/ether 34:98:b5:0d:97:0b brd ff:ff:ff:ff:ff:ff
9: phy1-ap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP group default qlen 1000
    link/ether 34:98:b5:0d:97:0c brd ff:ff:ff:ff:ff:ff
root@router:~#