Passive TAP / HUB and Ethernet Bridge

erdoukki · December 27, 2019, 5:43pm

I have no VLAN with EspressoBin Board...
So I may understand from your answer that a bridge of two ethernets is doing hardware switching, so it will stay silent from tcpdump...?
Is there any other bridging possibilities, without VLAN tagging, to accomplish wath I want ?
How can I do software switching of two ethernet like a bridge but still tcpdumpable ?

To say it another way, I want to be transparent (for a plug and play solution) looking and analysing all the network traffic between two subnets; here it is between my ISP routeur and my personnal OpenWRT routeur.

mk24 · December 27, 2019, 5:46pm

I don't know anything specific to the expressobin but the manufacturer's page says it has a gigabit switch with three Ethernet cable ports, and presumably one or two CPU ports also attached to the switch. Is that how it is built?

What I would do is set up three VLANs inside the switch, each untagged to one port and all tagged to the CPU port. Then you have exposed the untagged traffic (I assume you don't have to worry about VLANs on the cables) on the respective cables to eth0.1 eth0.2 and eth0.3. Attach one of those to the standard LAN so you can log in. Then make an unmanaged bridge holding the two others. In hardware, break in to the circuit you want to monitor by passing through those two ports.

erdoukki · December 27, 2019, 5:48pm

Yes, It is this like.

I want to use two of the ethernets cable ports for the "bridge" analysis.
The third is used to get the access for monitor / admin by ssh / luci to OpenWRT.

like said in the topaz description; http://wiki.espressobin.net/tiki-index.php?page=Topaz+Switch
The espressobin board may support vlan tagging, but in fact it is still not supported in OpenWRT.

mk24 · December 27, 2019, 5:59pm

If the switch only operates unmanaged, you can't do this application. It is just going to hardware switch the packets between the active cable ports and the CPU has no chance to see them.

erdoukki · December 27, 2019, 6:04pm

Okay, so it is impossible like I am trying to...
If the switch management is added to OpenWRT I can retry again with VTAG...

Is there any other software only solution ?

erdoukki · December 29, 2019, 5:43pm

I have added ip-bridge and then I can look at the bridge...

It is a DSA switch (so VLAN supported)
I am still in OpenWRT 18.06.2

root@OpenWrt:~# bridge link
3: lan1 state UP @eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 master br-br state forwarding priority 32 cost 19 
4: lan0 state LOWERLAYERDOWN @eth0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 master br-br state disabled priority 32 cost 100

I can see VLAN ids support;

root@OpenWrt:~# bridge vlan
port	vlan ids
lan1	None
lan0	None
br-br	None

I also read about a hwmode of bridge here : http://manpages.ubuntu.com/manpages/trusty/man8/bridge.8.html

I still a lot confused and cannot get working "like I need"...
So I reopen the post (and will close the other thread I have opened about this topic)

thanks...in advance

erdoukki · December 29, 2019, 6:34pm

I already view multicast IP with tcpdump from the bridged lan0 and lan1 as from the bridge itself...
But still no success with IP...

erdoukki · December 29, 2019, 8:04pm

root@OpenWrt:~# bridge vlan add vid 100 dev lan0
RTNETLINK answers: Not supported

Am I missing something about how to change the VLAN of my ports ?
Did the hardware okay ?

root@OpenWrt:~# dmesg |grep switch
[    0.680396] mv88e6085 d0032004.mdio-mii:01: switch 0x3410 detected: Marvell 88E6341, revision 0
[    0.697125] libphy: /soc/internal-regs@d0000000/mdio@32004/switch0@1/mdio: probed
[    1.026535] mv88e6085 d0032004.mdio-mii:01: switch 0x3410 detected: Marvell 88E6341, revision 0
[    1.046372] libphy: /soc/internal-regs@d0000000/mdio@32004/switch0@1/mdio: probed
[    1.060877] DSA: switch 0 0 parsed
[    1.808199] Marvell 88E6390 !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11: attached PHY driver [Marvell 88E6390] (mii_bus:phy_addr=!soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:11, irq=POLL)
[    1.938198] Marvell 88E6390 !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12: attached PHY driver [Marvell 88E6390] (mii_bus:phy_addr=!soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:12, irq=POLL)
[    2.068199] Marvell 88E6390 !soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13: attached PHY driver [Marvell 88E6390] (mii_bus:phy_addr=!soc!internal-regs@d0000000!mdio@32004!switch0@1!mdio:13, irq=POLL)

erdoukki · December 31, 2019, 12:03pm

Just tryed another espressobin board with a 19.07-rc2, and the VLAN tagging is bow working :

root@OpenWrt:~# bridge vlan
port	vlan ids
lan1	 1 PVID Egress Untagged
	 200

lan0	 1 PVID Egress Untagged
	 100

br-lan	 1 PVID Egress Untagged

changed vlan with :

bridge vlan add vid 100 dev lan0
bridge vlan add vid 200 dev lan1

Any advice will be welcome, now that I can play with VLAN tagging

erdoukki · December 31, 2019, 12:23pm

All works fine now, wireshark see all frames (UPD and IP) and tcpdump also view all traffic...

root@OpenWrt:~# opkg install bridge ip-bridge

VLAN DIFF

root@monitor:~# bridge vlan add vid 200 dev lan1 pvid master
root@monitor:~# bridge vlan add vid 100 dev lan0 pvid master

BRIDGE VLAN

root@monitor:~# bridge vlan
port	vlan ids
lan1	 1 Egress Untagged
	 200 PVID

lan0	 1 Egress Untagged
	 100 PVID

br-lan	 1 PVID Egress Untagged

BRIDGE MONTOR

bridge monitor

TCPDUMP

root@monitor:~# tcpdump -i br-lan

erdoukki · January 8, 2020, 7:41am

root@monitor:~# bridge vlan
port	vlan ids
lan1	 200 PVID Egress Untagged

lan0	 100 PVID Egress Untagged

br-lan	 1 PVID Egress Untagged

all is fine with tcpdump, but not with ulogd while using NFCT plugin !...

erdoukki · January 8, 2020, 8:26am

Still cannot ulog from my bridge...

root@monitor:~# ebtables -F

root@monitor:~# ebtables -L
Bridge table: filter

Bridge chain: INPUT, entries: 0, policy: ACCEPT

Bridge chain: FORWARD, entries: 0, policy: ACCEPT

Bridge chain: OUTPUT, entries: 0, policy: ACCEPT

root@monitor:~# ebtables -A FORWARD --ulog -j DROP
Unable to update the kernel. Two possible causes:
1. Multiple ebtables programs were executing simultaneously. The ebtables
   userspace tool doesn't by default support multiple ebtables programs running
   concurrently. The ebtables option --concurrent or a tool like flock can be
   used to support concurrent scripts that update the ebtables kernel tables.
2. The kernel doesn't support a certain ebtables extension, consider
   recompiling your kernel or insmod the extension.
.

https://ebtables.netfilter.org/examples/basic.html#ex_ulog
https://www.systutorials.com/docs/linux/man/8-ebtables/

erdoukki · January 8, 2020, 9:04am

root@monitor:~# echo "nfnetlink_log" >/proc/sys/net/netfilter/nf_log/7 
root@monitor:~# cat /proc/net/netfilter/nf_log 
 0 NONE (nfnetlink_log)
 1 NONE (nfnetlink_log)
 2 nf_log_ipv4 (nf_log_ipv4,nfnetlink_log)
 3 NONE (nfnetlink_log)
 4 NONE (nfnetlink_log)
 5 NONE (nfnetlink_log)
 6 NONE (nfnetlink_log)
 7 nfnetlink_log (nfnetlink_log)
 8 NONE (nfnetlink_log)
 9 NONE (nfnetlink_log)
10 nf_log_ipv6 (nf_log_ipv6,nfnetlink_log)
11 NONE (nfnetlink_log)
12 NONE (nfnetlink_log)

Still the same problem of silent TCP on my bridge and the lan0/lan1 bridged...

erdoukki · January 9, 2020, 5:45am

I made a second try with this documentation : https://www.kernel.org/doc/html/latest/networking/dsa/configuration.html and get some more success...
I can set the vlan_filtering only by sysctl:

root@monitor:~# echo "1">/sys/class/net/br0/bridge/vlan_filtering
root@monitor:~# cat /sys/class/net/br0/bridge/vlan_filtering 1

Now I get tcpdump only of IGMP, STP and some IPv6 (neightbour), some UDP (DNS:53)...
But still nothing from the IPv4 stack of what pass through the tunnel...
The traffic is correctly tunneled, but still silent !

I am stuck in this, testing all combinations possible...
Do I need IPT, EBT filter ?
Do I need some sysctl tweaking ?

thanks for the help...

erdoukki · January 9, 2020, 10:10am

Looks like my conntracking is faulty ?

root@monitor:~# conntrack -E
[DESTROY] udp      17 src=10.2.1.166 dst=51.15.191.239 sport=36478 dport=123 pac
kets=0 bytes=0 src=51.15.191.239 dst=10.2.1.166 sport=123 dport=36478 packets=0 
bytes=0
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34840 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34840
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34840 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34840 pack
ets=1 bytes=40
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34842 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34842
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34842 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34842 pack
ets=1 bytes=40
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34844 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34844
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34844 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34844 pack
ets=1 bytes=40
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34846 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34846
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34846 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34846 pack
ets=1 bytes=40
    [NEW] udp      17 60 src=fe80::f2ad:4eff:fe08:ab43 dst=ff02::1:2 sport=546 d
port=547 [UNREPLIED] src=ff02::1:2 dst=fe80::f2ad:4eff:fe08:ab43 sport=547 dport
=546
    [NEW] udp      17 60 src=fe80::f2ad:4eff:fe08:aaca dst=fe80::f2ad:4eff:fe08:
ab43 sport=547 dport=546 [UNREPLIED] src=fe80::f2ad:4eff:fe08:ab43 dst=fe80::f2a
d:4eff:fe08:aaca sport=546 dport=547
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34848 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34848
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34848 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34848 pack
ets=1 bytes=40
    [NEW] tcp      6 120 SYN_SENT src=10.2.1.111 dst=10.2.1.166 sport=34850 dpor
t=667 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34850
[DESTROY] tcp      6 src=10.2.1.111 dst=10.2.1.166 sport=34850 dport=667 packets
=1 bytes=60 [UNREPLIED] src=10.2.1.166 dst=10.2.1.111 sport=667 dport=34850 pack
ets=1 bytes=40

TCP stay UNREPLIED ???

while adding pseudo VLAN, tcpdump become MASS verbose... but still no connection track of IPv4...

# bridge vlan add vid 200 dev lan1 pvid untagged master
# bridge vlan add vid 100 dev lan0 pvid untagged master

root@monitor:~# bridge vlan
port    vlan ids
lan1     200 PVID Egress Untagged
         500 Egress Untagged

lan0     100 PVID Egress Untagged
         500 Egress Untagged

br-lan   500 PVID

I have put all my network interface in promisc mode and also add ipt call on the bridge with

sysctl net.bridge.bridge-nf-call-iptables=1
sysctl net.bridge.bridge-nf-call-ip6tables=1

anon50098793 · January 9, 2020, 10:22am

conntrack is too high in the protocol stack.... and only present for local and routed packets that pass "through" the router... the unreplied entries seem to be un-connected voip? ( i.e. the client probably falls back to a udp connection when tcp fails? )

( sorry i'm a visual person... i take it this device is inline between your WAN port and an NTD ( isp-modem or similar )

is it a gigabit link?

erdoukki · January 9, 2020, 3:30pm

So I cannot hope off listening with conntrack the bridged traffic ?

It's between my switch and my OpenWRT Internet Routeur...

Yes

anon50098793 · January 9, 2020, 3:47pm

try monitoring one end of a veth pair... with the other end in the bridge

erdoukki · January 9, 2020, 4:02pm

Can you tell more please ?
I am not sure to understand your proposal...

anon50098793 · January 9, 2020, 4:23pm

something like this...

opkg install kmod-veth tc kmod-dummy

sourceIF=br-lan
destIF=dummy0

tc qdisc add dev "$sourceIF" ingress
tc filter add dev "$sourceIF" parent ffff: \
          protocol all \
          u32 match u8 0 0 \
          action mirred egress mirror dev "$destIF"

tc qdisc add dev "$sourceIF" handle 1: root prio
tc filter add dev "$sourceIF" parent 1: \
          protocol all \
          u32 match u8 0 0 \
          action mirred egress mirror dev "$destIF"

ifconfig $destIF up
tcpdump -elni $destIF

or this... for veth

destIF=vethD
tapIF=vethS
ip link add $tapIF type veth peer name $destIF
brctl addif br-lan $tapIF
ifconfig $tapIF up
ifconfig $destIF up