Mini tutorial for DSA network config

Possible, yes, but I cannot verify since I don't have the hardware.
Did you load the required kernel modules (check with lsmod)?

1 Like

have add two ipk :

root@ultra:~# opkg install kmod-sched
root@ultra:~# opkg install kmod-sched-act-vlan

lsmod shows necessary kernel modules :

root@ultra:~# lsmod |grep sch_ingress
sch_ingress            12288  2 
root@ultra:~# lsmod |grep cls_matchall
cls_matchall           12288  0 
root@ultra:~# lsmod |grep act_mirred
act_mirred             12288  6 

but still not working...

Have take a look to the blog referred in the link you give...
but still no success (for now) :wink:

https://man7.org/linux/man-pages/man8/tc-matchall.8.html

   skip_sw
          Do not process filter by software. If hardware has no
          offload support for this filter, or TC offload is not
          enabled for the interface, operation will fail.

   skip_hw
          Do not process filter by hardware.
service network restart

ip link set up dev br-tap
tc qdisc add dev br-lan clsact
tc filter add dev br-lan ingress matchall skip_hw action mirred egress mirror dev br-tap
tc filter add dev br-lan egress matchall skip_hw action mirred egress mirror dev br-tap

but still nothing on the mirror !?

root@ultra:~# tcpdump -n -i br-tap
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on br-tap, link-type EN10MB (Ethernet), capture size 262144 bytes
service network restart
ip link set up dev br-tap
tc qdisc add dev br-lan clsact
tc filter add dev br-lan egress matchall skip_hw action mirred ingress mirror dev br-tap
tc filter add dev br-lan ingress matchall skip_hw action mirred egress mirror dev br-tap
root@ultra:~# tc -s -p qdisc ls dev br-lan
qdisc noqueue 0: root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc clsact ffff: parent ffff:fff1 
 Sent 3151 bytes 41 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
root@ultra:/# tcpdump -n -i br-tap
[ 2104.580420] device br-tap entered promiscuous mode                                                                                               
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode                                                                          
listening on br-tap, link-type EN10MB (Ethernet), capture size 262144 bytes                                                                         
15:55:28.555826 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 259, length 64                                                               
15:55:29.558440 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 260, length 64                                                               
15:55:30.558757 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 261, length 64                                                               
15:55:31.560134 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 262, length 64                                                               
15:55:32.560918 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 263, length 64                                                               
15:55:33.562084 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 264, length 64                                                               
15:55:34.563810 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 265, length 64                                                               
15:55:35.564057 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 266, length 64                                                               
15:55:36.565928 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 267, length 64                                                               
15:55:37.567172 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 268, length 64                                                               
15:55:38.569459 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 269, length 64                                                               
15:55:39.568960 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 270, length 64                                                               
15:55:40.570508 IP 10.4.2.224 > 10.4.2.112: ICMP echo reply, id 3, seq 271, length 64                                                               

WHOOO... There were a little bug in the tutorial given !?
I have mirrored ingress to egress and egress to ingress...
now it talks... (a lot !)
I hope I have done it correctly ???

sorry for the little out of topic...

thanks

edited: I have to cross egree/ingress, to not getting DUP in PING command...
Hope I have done the mirror correctly ???

I have to use the console to not get loop of my own traffic when logged by ssh...?

Not sure if bridge devices are supported as source or destination. I guess it is called port mirror for a reason.
Try with lan1, lan2 or similar.

The tutorial looks correct to me, both should be mirrored to egress of the monitoring port.

https://biot.com/switches/testing/mirroring

It's needed packages ip-full and tc-full.

1 Like
root@ultra:/# opkg install tc-full ip-full                                                                                                          
Package tc-full (5.11.0-3) installed in root is up to date.                                                                                         
Package ip-full (5.11.0-3) installed in root is up to date.                                                                                         

still silent if not "crossed" to lan0...

okay tried with wlan0 and wan, to (mirror) lan0... still silent !

But I get to change skip_sw by skip_hw ! if not I get error with kernel...

thanks for the script...
I only had to modify skip_sw to skip_hw , and set mirror port to lan0.
But still silent on my tcpdump -i lan0

@eduardo010174
How do you "listen" to your mirror port ?
With tcpdump -i lan2 ?

#!/bin/sh

#creds https://biot.com/switches/testing/mirroring

sniffPort=wan
mirrorPort=lan0

#Turn on the mirror port
ip link set up dev ${mirrorPort}

#Add the clsact queue discipline. This qdisc lets us attach the matchall filter
tc qdisc add dev ${sniffPort} clsact

# enable hw tc offload.
ethtool -K ${sniffPort} hw-tc-offload on

#Mirror all packets inbound on sniffPort (ingress) to mirrorPort. Note the skip_sw flag, meaning this command will not fall back on mirroring via the CPU if the hardware offload fails
tc filter add dev ${sniffPort} ingress matchall skip_sw action mirred egress mirror dev ${mirrorPort}

#Mirror all packets going out of snifPort (egress) to mirrorPort
tc filter add dev ${sniffPort} egress matchall skip_sw action mirred egress mirror dev ${mirrorPort}

REF : https://patchwork.ozlabs.org/project/netdev/patch/20160704073411.17633-1-amir@vadai.me/

I have modified to use the skip_sw flags by enabling the hardware offload, no more kernel error, but still silent with tcpdump...

root@ultra:~# tc -s -p qdisc ls dev wan
qdisc noqueue 0: root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
qdisc clsact ffff: parent ffff:fff1 
 Sent 199423 bytes 1345 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0
root@ultra:~# tc -s -p qdisc ls dev lan0
qdisc noqueue 0: root refcnt 2 
 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) 
 backlog 0b 0p requeues 0

according to this : https://www.linux.org/docs/man8/tc-mirred.html
this script will work for me :

root@ultra:~# cat ipmirror.sh

#!/bin/sh

#https://www.linux.org/docs/man8/tc-mirred.html
#https://medium.com/swlh/traffic-mirroring-with-linux-tc-df4d36116119

sniffPort=wan
mirrorPort=dummy0

#Turn on the mirror port
ip link add ${mirrorPort} type dummy
ip link set ${mirrorPort} up

#Add the ingress queue discipline. This qdisc lets us attach the matchall filter
##tc qdisc add dev ${sniffPort} handle ffff: ingress
tc qdisc del dev ${sniffPort} handle ffff: ingress
tc qdisc add dev ${sniffPort} handle ffff: ingress

#Mirror all incoming ICMP packets on sniffPort to a mirrorPort interface for examination with e.g. tcpdump:
##tc filter add dev ${sniffPort} parent ffff: protocol ip u32 match ip protocol 1 0xff action mirred egress mirror dev ${mirrorPort}

#Mirror all incoming IP packets on sniffPort to a mirrorPort interface for examination with e.g. tcpdump:
##tc filter add dev ${sniffPort} parent ffff: protocol ip u32 match u32 0 0 action mirred egress mirror dev ${mirrorPort}

#Mirror all incoming traffic on sniffPort to a mirrorPort interface for examination with e.g. tcpdump:
tc filter add dev ${sniffPort} parent ffff: protocol all u32 match u32 0 0 action mirred egress mirror dev ${mirrorPort}

tc qdisc del dev ${sniffPort} handle 1: root
tc qdisc add dev ${sniffPort} handle 1: root prio

#Mirror all outgoing traffic from sniffPort to a mirrorPort interface for examination with e.g. tcpdump:
tc filter add dev ${sniffPort} parent 1: protocol all u32 match u32 0 0 action mirred egress mirror dev ${mirrorPort}

tc -s -p filter ls dev ${sniffPort} parent ffff:

tc -s qdisc ls dev ${sniffPort}

echo tcpdump -n -i ${mirrorPort}

edited : fixed IP

Thank you @rmilecki for great tutorial. I am just wondering what is the difference between lan4 and lan4:u in VLAN scenario, and what is the effect of *?
While lan4:t will tag traffic leaving port, both lan4 and lan4:u shouldn't tag traffic.
Also some devices (R7800 for example) have 'CPU' switch ports (eth0 and eth1 in that case). Should those be somehow assigned to bridge(s)?

Current possibilities I think:

Syntax Member Port Is
lanx untagged ~PVID
lanx:u untagged
lanx:t tagged
lanx:* PVID untagged
lanx:u* PVID untagged
lanx:t* PVID tagged

So if I would be migrating following access point config with WAN port used as trunk to the router:


/cut/
config interface 'lan'
	option type 'bridge'
	option proto 'static'
	option netmask '255.255.255.0'
	option ipv6 '0'
	option ipaddr '10.0.0.2'
	option gateway '10.0.0.1'
	option hostname 'ap'
	option ifname 'eth1.1'
	option delegate '0'

config switch
	option name 'switch0'
	option reset '1'
	option enable_vlan '1'

config switch_vlan
	option device 'switch0'
	option vlan '1'
	option vid '1'
	option description 'MAIN-VLAN'
	option ports '6t 4 3 2 1 5t'

config switch_vlan
	option device 'switch0'
	option vlan '3'
	option vid '3'
	option description 'GUEST-VLAN'
	option ports '0t 5t'

config interface 'guest'
	option proto 'none'
	option ifname 'eth1.3'1
	option type 'bridge'

DSA config should look like:

/cut/
config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	list ports 'wan'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:u*'
	list ports 'lan2:u*'
        list ports 'lan3:u*'
	list ports 'lan4.1:t*'
        list ports 'wan.1:t*'

config bridge-vlan          
        option device 'br-lan'
        option vlan '3'       
        list ports 'lan4.3:t'
        list ports 'wan.3:t'   


config interface 'lan'
	option device 'br-lan.1'
	option proto 'static'
	option ipaddr '10.0.0.2'
	option netmask '255.255.255.0'
	option ip6assign '60'

config interface 'guest'
	option device 'br-lan.3'
	option proto 'none'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '1'
	option name 'br-lan.1'

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '3'
	option name 'br-lan.3'

A number of the possibilities from the table I posted are redundant / superfluous, but legitimate. Some of your syntax looks to be incorrect. Here is some stuff off a device which sets up correctly, but is really just a test device, not meant to be a running system, but shows how things manifest:

example
root@mamba:/etc/config# cat network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option packet_steering '1'
	option ula_prefix 'auto'

config device
	option name 'wan'
	option macaddr '11:22:33:44:55:66'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config interface 'wan6'
	option device 'wan'
	option proto 'dhcpv6'

config device
	option name 'lan1'
	option macaddr '11:22:33:44:55:66'

config device
	option name 'lan2'
	option macaddr '11:22:33:44:55:66'

config device
	option name 'lan3'
	option macaddr '11:22:33:44:55:66'

config device
	option name 'lan4'
	option macaddr '11:22:33:44:55:66'

config device
	option name 'itch0'
	option type 'bridge'
	list ports 'lan4'
	list ports 'lan3'
	list ports 'lan2'
	list ports 'lan1'
	option vlan_filtering '1'
#	option igmp_snooping '1'
#	option stp '1'

config bridge-vlan
	option device 'itch0'
	option vlan '10'
	list ports 'lan4:*'
	list ports 'lan3'
#	list ports 'lan2'
#	list ports 'lan1'
	option alias 'lan'

config interface 'lan'
	option proto 'static'
	option ipaddr '192.168.10.1'
	option netmask '255.255.255.0'
	option ip6assign '60'
#	option ip6assign '56'
#	option ip6hint '10'
	option device 'itch0.lan'

config bridge-vlan
	option device 'itch0'
	option vlan '20'
	list ports 'lan2:t'
	list ports 'lan1:t'
	option alias 'ap'

config interface 'ap'
	option proto 'static'
	option ipaddr '192.168.20.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option device 'itch0.ap'

config bridge-vlan
	option device 'itch0'
	option vlan '30'
	list ports 'lan2:t'
	list ports 'lan1:u*'
	option alias 'iot'

config interface 'iot'
	option proto 'static'
	option ipaddr '192.168.30.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option device 'itch0.iot'

config bridge-vlan
	option device 'itch0'
	option vlan '4094'
	list ports 'lan2:t*'
	option alias 'sink'

config interface 'windscribevpn'
	option proto 'none'
	option device 'tun0'

root@mamba:/etc/config# bridge vlan
port              vlan-id  
lan4              10 PVID Egress Untagged
lan3              10 PVID Egress Untagged
lan2              20
                  30
                  4094 PVID
lan1              20
                  30 PVID Egress Untagged
itch0             10
                  20
                  30
                  4094
wlan0             10 PVID Egress Untagged
wlan1             10 PVID Egress Untagged
wiot0             30 PVID Egress Untagged
wiot1             30 PVID Egress Untagged
root@mamba:/etc/config# netifd-netinfo.sh -d
           DEVICE     UP    CARRIER    PRESENT   EXTERNAL  TYPE             
=============================================================================
             eth0      x          x          x             Network device   
            itch0      x          x          x             bridge           
         itch0_10      x          x          x             VLAN             
         itch0_20      x          x          x             VLAN             
         itch0_30      x          x          x             VLAN             
             lan1      x                     x             Network device   
             lan2      x                     x             Network device   
             lan3      x          x          x             Network device   
             lan4      x          x          x             Network device   
               lo      x          x          x             Network device   
              wan      x          x          x             Network device   
            wiot0      x          x          x          x  Network device   
            wiot1      x          x          x          x  Network device   
            wlan0      x          x          x          x  Network device   
            wlan1      x          x          x          x  Network device   

My network file:

lanx with nothing -> lanx untagged

lanx:t -> lanx tagged

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	list ports 'wan'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:t'
	list ports 'wan'

config bridge-vlan
	option device 'br-lan'
	option vlan '100'
	list ports 'lan1:t'
	list ports 'lan4'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'static'
	option ipaddr '192.168.0.162'
	option gateway '192.168.0.1'
	option netmask '255.255.255.0'
	list dns '192.168.0.1'

config device
	option type 'bridge'
	list ports 'br-lan.100'
	option name 'TV'
	option igmpversion '2'
1 Like

Thanks @anomeome and @anon69880279 for provided examples. With their help I've migrated both R7800s to DSA config while preserving VLAN trunking.
Small remark: R7800 DSA driver seams to number switch ports in the opposite direction to actual port numbers on the device.
BTW OpenWrt with DSA is not having on AX3600 issue with 802.1q trunk MTU size limitation compared to stock firmware.
Still I am struggling with one thing: DHCP. If DHCP is enabled on R7800 configured as a router only wireless clients can obtain configuration. Wired are waiting into infinity for their IP addresses. However if I move exactly the same DHCP config to R7800 configured as access point everything is working fine for both wired and wireless cients with an exception of those connected with Ethernet directly to router R7800. It also worked fine before move to DSA.
Tried so far on the router turning off firewall/bcp38 and banIP. I've ran out of ideas (except of troubleshooting with packet sniffer and finding a way to enable logging for DHCP in dnsmasq).
Below router (not working) config:

Summary
/cut/
config globals 'globals'
	option packet_steering '1'

config device
	option name 'wan'
/cut/
config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan4'
	list ports 'lan3'
	list ports 'lan2'
	list ports 'lan1'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4:t'
	list ports 'tap0'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'lan4:t'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'static'
	option netmask '255.255.255.0'
	option ipaddr '10.0.0.1'
	option metric '0'
	option delegate '0'

config interface 'guest'
	option proto 'static'
	option ipaddr '192.168.3.1'
	option netmask '255.255.255.0'
	option device 'br-lan.3'

config device
	option name 'br-lan.1'
	option type '8021q'
	option ifname 'br-lan'
	option vid '1'

config device
	option type '8021q'
	option name 'br-lan.3'
	option vid '3'
	option ifname 'br-lan'

and /etc/config/dhcp

Summary
config dhcp 'lan'
	option interface 'lan'
	option limit '150'
	option leasetime '12h'
	list dhcp_option_force '42,10.0.0.1'
	list dhcp_option '42,10.0.0.1'
	option force '1'
	option start '10'
	list ra_flags 'none'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'guest'
	option start '100'
	option leasetime '12h'
	option limit '150'
	option interface 'guest'
	list ra_flags 'none'

Can someone share DHCP config (/etc/config/dhcp) that works with DSA? Especially with wireless and ethernet bridged together under one VLAN?

I don't suppose someone has an example guest network config for a WRT 3200acm? I have tried this configuration:

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'lan4:u*'

config interface 'lan'
	option device 'br-lan.1'
	option proto 'static'
	option ipaddr '192.168.1.1'
	option netmask '255.255.255.0'

config interface 'guest'
	option device 'br-lan.2'
	option proto 'static'
	option ipaddr '192.168.5.1'
	option netmask '255.255.255.0'

But it seemed to hang my router. Sorry I'm a bit of a newb to DSA. I had it setup previously using swconfig.

Yes, I wish the examples included multiple wireless networks. It is non-obvious how to transform them from the simple single network to multiple ssids.

Warning: I'm uncertain if the example below is the CORRECT WAY. All I know is that it Works For Me(tm). Hoping some expert will chime in here and correct me...

IIUC, you still need to create a separate bridge for each wireless network. Which used to be indicated with a option type 'bridge' on the affected interfaces. This should now be explicitly created with a config device section.

This is an example from a Unifi 6 Lite, which has a single ethernet port named "lan" (very confusing!) on the embedded MT7621/MT7530 switch:

root@u6-1:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option packet_steering '1'
        option ula_prefix 'fd62:fdd2:da95::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'lan.10'

config device
        option name 'br-iot'
        option type 'bridge'
        list ports 'lan.15'

config interface 'lan'
        option device 'br-lan'
        option proto 'none'

config interface 'iot'
        option device 'br-iot'
        option proto 'none'

config interface mgmt
        option device lan.203
        option proto 'static'
        option ipaddr '192.168.99.55'
        option netmask '255.255.255.0'
        option ip6assign '60'
        list dns '192.168.99.3'

I have two wireless networks here - "lan" and "iot" - connected to VLAN IDs 10 and 15 respectively. The access point has no IP address on any of these. They could have had, but I don't need it. You'd obviously jjust replace the option proto 'none' wtuff with whatever you want.

In addition there is a wired only VLAN ID 203 which I use for access point management. This is wired only, so there is no need to create a bridge for it.

All three VLANs are tagged on the only ethernet port.

Note that the port and device references to lan.x refers to a physical ethernet port on the switch named "lan", while the interface reference to lan refers to the netifd virtual interface name of the br-lan device. Yes, it is extremely confusing... I didn't choose the name of that switch port.

Some output to help illustrate:

root@u6-1:~# brctl show
bridge name     bridge id               STP enabled     interfaces
br-iot          7fff.f492bfac9194       no              wlan0-1
                                                        lan.15
br-lan          7fff.f492bfac9194       no              wlan0
                                                        lan.10
                                                        wlan1
root@u6-1:~# bridge vlan
port              vlan-id  
br-iot            1 PVID Egress Untagged
lan.15            1 PVID Egress Untagged
br-lan            1 PVID Egress Untagged
lan.10            1 PVID Egress Untagged
wlan0             1 PVID Egress Untagged
wlan0-1           1 PVID Egress Untagged
wlan1             1 PVID Egress Untagged
3 Likes

What is the purpose of having vlan 2 on lan4 untagged?

1 Like