Iptables-mod-tee to monitor outgoing traffic from a WAN interface

Installing and Using OpenWrt
1

I have OpenWrt v18.06.5, r7897-9d401013fc loaded on the Netgear R7800. This router does a standard (out of the box) NAT between a 192.168.0.1/24 LAN (on eth1 ) and one public IP 140.82.118.4 (on eth2 ). The router has the iptables-mod-tee module loaded and running.

My goal is to monitor the traffic going OUT of the interface eth2 to the WAN. The monitoring is to be done by a computer running "Wireshark", which is located on the LAN (at 192.168.0.3 ).

For example when I ping 8.8.8.8 from another computer on the LAN (e.g. from 192.168.0.2 ), I expect to see an IP/ICMP Echo request, that has ip.src == 140.82.118.4 and ip.dst == 8.8.8.8 going out on the eth2 interface to the WAN.

I am using the following command to modify the iptables to send the cloned packets to the host running the "Wireshark":
iptables -t mangle -A POSTROUTING -o eth2 -j TEE --gateway 192.168.0.3

It works but "Wireshark" displays the IP/ICMP Echo request with ip.src == 192.168.0.2 and ip.dst == 8.8.8.8 ...which is wrong, because the ip.src looks like it would be before it is NAT'ed !!!

The ip.src should be 140.82.118.4 which is the IP of my public WAN interface... because if it wasn't, then the 8.8.8.8 host would not know where to send the ICMP Echo reply to, however the ping command is getting ICMP Echo replies from the 8.8.8.8 host (as it should).

Why is this happening and how to correct it ?

P.S.
Also, where is the documentation how the POSTROUTING and the -o options work ?

406×842

2

Personally, I think I'd make your mirroring port part of the WAN VLAN - and then configure "port mirroring" on the swconfig level.

1 Like
3

Are you capturing from a lan device or the router itself using tcpdump?

4

@mindwolf
I am not using tcpdump. I am using iptables-mod-tee.
Please take a look at the diagram - I am capturing the cloned packets on a LAN host (at 192.168.0.3) which is running "Wireshark".

5

I saw that, but was wondering if you did a live capture on your router first, then analyzed it on your computer. If capturing from the lan behind the router, I would venture to say you would not see the external ip address but the local ip instead.Thus you didn't cross the bridge...PUN INTENDED :wink:

6

Capture
Capture944×84 4.47 KB

7

I did try sing tcpdump and it works as expected, namely it shows ip.src == 140.82.118.4.
However using tcpdump on the router does not satisfy my real-time monitoring needs, as I will have custom software doing this analysis on the LAN host (at 192.168.0.3)

Why?
When I look at the iptables diagram (attached below) then it seems that the "mangle table" with "POSTROUTING chain" would have the ip.src already translated to the IP of the eth2 (wan IP 140.82.118.4) because it is after the "filter table" with the "FORWARD chain" and after the "NAT table" with the "OUTPUT chain".

Only the "NAT table" with the "POSTROUTING chain" is closer to the outbound WAN interface (eth2) than the "mangle table"...but can iptables-mod-tee do the packet cloning there ?

iptables
iptables2148×1108 402 KB

8

Are you SURE that your wan egress interface is eth2? Normally OpenWRT is setup with a VPID or simply a virtual interface with vlans' attached.

1 Like
9

No it isn't. The interface names are randomized.

10

what does your iptables -t mangle show?

11 
root@OpenWrt:~# iptables --list -t mangle

Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination

Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination
TCPMSS     tcp  --  anywhere             anywhere             tcp flags:SYN,RST/SYN /* !fw3: Zone wan MTU fixing */ TCPMSS clamp to PMTU

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
TEE        all  --  anywhere             anywhere             TEE gw:192.168.0.3
12

Could this be the problem ?

root@OpenWrt:~# iptables --list -t nat

<...snip...>

Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
zone_wan_postrouting  all  --  anywhere             anywhere             /* !fw3 */

<...snip...>

Chain zone_wan_postrouting (1 references)
target     prot opt source               destination
MASQUERADE  all  --  anywhere             anywhere             /* !fw3 */

WARNING: This was not the complete output of the iptables --list -t nat command.
I deleted irrelevant stuff where I've marked it with <...snip...>

13

Yes, the MASQUERADE target in the nat:POSTROUTING chain was the problem.
I have found a solution to cloning / monitoring the snat'ed outbound traffic on the WAN interface (eth2) with the TEE (iptables-mod-tee) ...if anyone is interested.