Sometimes router does not route packages

I have following setup:

  • Main router which performs NAT, DHCP etc.
  • Roaming system which function as bridge between network and wireless networks at various points of my home

Sometimes when I connect via WiFi to roaming system the packages do not get back. So I can for example send ping to main router but it does not reply. I can ssh into roaming system and ping router from there so the connection is fine.

I think from previous digging into it that router does send a reply to bridge but it does not appear on the port that it should get through. I suspect there is some caching of MAC addresses going on but I have no idea how to fix it.

You will need to provide much more information in order to get any meaningful help.

  • Are all the systems running OpenWrt? What version?
  • What are the devices (brand/model), and how are they connected to each other (wired, wifi)? A diagram can be really helpful here (a photo of a sketch on paper is sufficient).
  • And how are they configured? (we'll probably want to see the config files, but for now, did you follow any tutorials? did you install any packages or do any special configs, etc.)?

Are all the systems running OpenWrt? What version?
What are the devices (brand/model)?

22.03.2 for main router (WRT3200ACM) and r22307 for roaming (Banana Pi R3).

and how are they connected to each other (wired, wifi)? A diagram can be really helpful here (a photo of a sketch on paper is sufficient).

They are all wired by 1G ethernet in star configuration. The router is spoke of star.

1G ethernet is split into 3 VLANs - 1 for my network, 1 for guests and 1 for IOT.

And how are they configured?

LAN configured as bridge over 4 ports. For example on router:

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:t'
	list ports 'lan2:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'lan1:t'
	list ports 'lan2:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'lan1:t'
	list ports 'lan2:t'

br-lan.1, br-lan.2 and br-lan.3 are members of separate bridges (this time with WiFi). Similar configuration is on roaming stations (this time with tagging on 'WAN' port).

let's take a look at the following files for each device... please make it clear which is the WRT3200ACM and which is the BPi R3:

Please copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall

I hope I haven't missed anything.

Router:

# cat /etc/config/network 
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix '<ULA_PREFIX1>/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:t'
	list ports 'lan2:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'lan1:t'
	list ports 'lan2:t'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '64'
	list ip6class 'wan6'
	option ip6weight '0'
	option ipaddr '<LAN_IP_PREFIX>.1'

config device
	option name 'wan'
	option macaddr '<WAN_MAC>'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config device
	option name 'br-lan-secure'
	option type 'bridge'
	list ports 'br-lan.3'

config interface 'LAN_SECURE'
	option device 'br-lan-secure'
	option proto 'static'
	option ipaddr '<LAN_SECURE_IP_PREFIX>.1'
	option netmask '255.255.255.0'
	option ip6weight '3'
	option ip6assign '64'
	list ip6class 'WAN6SECURE'

config device
	option type 'bridge'
	option name 'br-lan-iot'
	option bridge_empty '1'
	list ports 'br-lan.2'

config interface 'LAN_IOT'
	option proto 'static'
	option device 'br-lan-iot'
	option ipaddr '<LAN_IOT_IP_PREFIX>.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6weight '1'

config device
	option type 'bridge'
	option name 'br-lan-guest'
	option bridge_empty '1'
	list ports 'br-lan.1'

config interface 'LAN_GUEST'
	option proto 'static'
	option device 'br-lan-guest'
	option ipaddr '<LAN_GUEST_IP_PREFIX>.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6weight '2'
	list ip6class 'WAN6GUEST'

config device
	option type 'macvlan'
	option ifname 'wan'
	option mode 'vepa'
	option name 'wan-secure'
	option macaddr '<WAN_SECURE_MAC>'

config device
	option type 'macvlan'
	option ifname 'wan'
	option mode 'vepa'
	option name 'wan-guest'
	option macaddr '<WAN_GUEST_MAC>'

config interface 'WAN6SECURE'
	option proto 'dhcpv6'
	option device 'wan-secure'
	option reqaddress 'try'
	option reqprefix 'auto'

config interface 'WAN6GUEST'
	option proto 'dhcpv6'
	option device 'wan-guest'
	option reqaddress 'try'
	option reqprefix 'auto'

config interface 'WAN6IOT'
	option proto 'dhcpv6'
	option device 'wan-iot'
	option reqaddress 'try'
	option reqprefix 'auto'

config device
	option type 'macvlan'
	option ifname 'br-lan'
	option mode 'vepa'
	option name 'wan-iot'
	option macaddr '<WAN_IOT_MAC>'
# cat /etc/config/wireless 

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11a'
	option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
	option country 'US'
	option cell_density '0'
	option htmode 'VHT80'
	option channel '36'

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11g'
	option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
	option htmode 'HT20'
	option country 'US'
	option cell_density '0'
	option channel '6'
	option disabled '1'

config wifi-device 'radio2'
	option type 'mac80211'
	option channel '34'
	option hwmode '11a'
	option path 'platform/soc/soc:internal-regs/f10d8000.sdhci/mmc_host/mmc0/mmc0:0001/mmc0:0001:1'
	option htmode 'VHT80'
	option disabled '1'

config wifi-iface 'wifinet0'
	option device 'radio1'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option key '<IOT_PASSWORD>'
	option disassoc_low_ack '0'
	option macaddr '<ROUTER_WLAN2_MAC>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan1-iot'
	option disabled '1'

config wifi-iface 'wifinet4'
	option device 'radio0'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option key '<SECURE_PASSWORD>'
	option network 'LAN_SECURE'
	option macaddr '<ROUTER_WLAN5_MAC>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-secure'

config wifi-iface 'wifinet5'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT2_SSID>'
	option key '<IOT2_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option disabled '1'

config wifi-iface 'wifinet6'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option disassoc_low_ack '0'
	option key '<IOT_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-iot'

config wifi-iface 'wifinet7'
	option device 'radio0'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option key '<GUEST_PASSWORD>'
	option network 'LAN_GUEST'
	option wpa_disable_eapol_key_retries '1'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-guest'

config wifi-iface 'wifinet8'
	option device 'radio1'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option ifname 'wlan1-guest'
	option key '<GUEST_PASSWORD>'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_GUEST'
	option disabled '1'

config wifi-iface 'wifinet9'
	option device 'radio1'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option encryption 'psk2'
	option key '<SECURE_PASSWORD>'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_SECURE'
	option ifname 'wlan1-secure'
	option disabled '1'
# cat /etc/config/dhcp 

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	list address '/<HOST1_NAME>.local/<LAN_IOT_IP_PREFIX>.<HOST1_IP>'
	list address '/<HOST2_NAME>.local/<LAN_IOT_IP_PREFIX>.<HOST2_IP>'
	list server '1.1.1.1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'LAN_SECURE'
	option interface 'LAN_SECURE'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option force '1'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'

config dhcp 'LAN_GUEST'
	option interface 'LAN_GUEST'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option force '1'
	option ra 'server'
	option dhcpv6 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config host
	option name '<HOST1_NAME>'
	option dns '1'
	option mac '<HOST1_MAC>'
	option ip '<LAN_IOT_IP_PREFIX>.<HOST1_IP>'

config host
	option name '<HOST2_NAME>'
	option dns '1'
	option mac '<HOST2_MAC>'
	option ip '<LAN_IOT_IP_PREFIX>.<HOST2_IP>'

config dhcp 'LAN_IOT'
	option interface 'LAN_IOT'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option dhcpv6 'server'
	list ra_flags 'other-config'

config dhcp 'LAN'
	option interface 'LAN'
	option start '100'
	option limit '150'
	option leasetime '12h'

config host
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '<LAN_IP_PREFIX>.2'
	option name 'roaming1'

config host
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '<LAN_SECURE_IP_PREFIX>.2'
	option name 'roaming1'

config host
	option name 'roaming1'
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '<LAN_GUEST_IP_PREFIX>.2'

config host
	option name 'roaming1'
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '<LAN_IOT_IP_PREFIX>.2'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option ra_slaac '0'
	option dhcpv6 'server'

config host
	option dns '1'
	option ip '<LAN_IOT_IP_PREFIX>.3'
	option name 'roaming2'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming2'
	option dns '1'
	option ip '<LAN_GUEST_IP_PREFIX>.3'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming3'
	option dns '1'
	option ip '<LAN_SECURE_IP_PREFIX>.3'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming3'
	option dns '1'
	option ip '<LAN_IP_PREFIX>.3'
	option mac '<ROAMING2_MAC>'
# cat /etc/config/firewall 

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option name 'lan_iot'
	list network 'LAN_IOT'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'WAN6SECURE'
	list network 'WAN6GUEST'
	list network 'WAN6IOT'

config forwarding
	option dest 'wan'
	option src 'lan_iot'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option proto 'esp'
	option target 'ACCEPT'
	option dest 'lan_iot'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'
	option dest 'lan_iot'

config rule
	option name 'Support-UDP-Traceroute'
	option src 'wan'
	option dest_port '33434:33689'
	option proto 'udp'
	option family 'ipv4'
	option target 'REJECT'
	option enabled 'false'

config include
	option path '/etc/firewall.user'

config zone
	option name 'lan_secure'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'LAN_SECURE'

config zone
	option name 'lan_guest'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'LAN_GUEST'

config forwarding
	option src 'lan_secure'
	option dest 'wan'

config forwarding
	option src 'lan_guest'
	option dest 'wan'

config forwarding
	option src 'lan_secure'
	option dest 'lan_iot'

config rule
	option name 'Allow-IOT-mDNS'
	list proto 'udp'
	list src_ip '224.0.0.251'
	option src_port '5353'
	list dest_ip '224.0.0.251'
	option dest_port '5353'
	option target 'ACCEPT'
	option family 'ipv4'
	option src 'lan_iot'

config rule
	option name 'Allow-IOT-mDNS6'
	option family 'ipv6'
	list proto 'udp'
	list dest_ip 'ff02::fb'
	option dest_port '5353'
	option target 'ACCEPT'
	option src 'lan_iot'

config zone
	option input 'ACCEPT'
	option output 'ACCEPT'
	option name 'lan'
	option forward 'ACCEPT'
	list network 'LAN'

config forwarding
	option src 'lan'
	option dest 'wan'

Roaming:

# cat /etc/config/network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix '<ULA_PREFIX2>/48'

config device
	option name 'br-lan'
	option type 'bridge'
	option macaddr '<ROAMING1_MAC>'
	option empty_bridge '1'
	list ports 'wan'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'wan:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'wan:t'
	list ports 'lan1:u'
	list ports 'lan2:u'
	list ports 'lan3:u'
	list ports 'lan4:u'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'wan:t'

config interface 'LAN'
	option device 'br-lan'
	option proto 'dhcp'

config device
	option name 'br-lan-guest'
	option type 'bridge'
	option empty_bridge 1
	list ports 'br-lan.1'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_GUEST'
	option device 'br-lan-guest'
	option proto 'dhcp'

config device
	option name 'br-lan-iot'
	option type 'bridge'
	option empty_bridge 1
	list ports 'br-lan.2'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_IOT'
	option device 'br-lan-iot'
	option proto 'dhcp'

config device
	option name 'br-lan-secure'
	option type 'bridge'
	list ports 'br-lan.3'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_SECURE'
	option device 'br-lan-secure'
	option proto 'dhcp'
# cat /etc/config/wireless 

# 2.4 GHz WiFi is disabled on roaming2
config wifi-device 'radio0'
	option type 'mac80211'
	option path 'platform/soc/18000000.wmac'
	option band '2g'
	option htmode 'HE20'
	option channel '6'
	option cell_density '0'

config wifi-device 'radio1'
	option type 'mac80211'
	option path 'platform/soc/18000000.wmac+1'
	option band '5g'
	option cell_density '0'
	option htmode 'HE80'
	option channel '36'

config wifi-iface 'wifinet0'
	option device 'radio1'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option key '<IOT_PASSWORD>'
	option disassoc_low_ack '0'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'

config wifi-iface 'wifinet4'
	option device 'radio0'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option key '<SECURE_PASSWORD>'
	option network 'LAN_SECURE'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'

config wifi-iface 'wifinet6'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option disassoc_low_ack '0'
	option key '<IOT_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'

config wifi-iface 'wifinet7'
	option device 'radio0'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option key '<GUEST_PASSWORD>'
	option network 'LAN_GUEST'
	option wpa_disable_eapol_key_retries '1'

config wifi-iface 'wifinet8'
	option device 'radio1'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option ifname 'wlan1-guest'
	option key '<GUEST_PASSWORD>'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_GUEST'

config wifi-iface 'wifinet9'
	option device 'radio1'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option encryption 'psk2'
	option key '<SECURE_PASSWORD>'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_SECURE'
# cat /etc/config/dhcp 

config dnsmasq
	option domainneeded '1'
	option boguspriv '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option nonegcache '0'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option nonwildcard '1'
	option localservice '1'
	option ednspacket_max '1232'
	option filter_aaaa '0'
	option filter_a '0'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	option ra_slaac '1'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'
# cat /etc/config/firewall 
config defaults
	option syn_flood	1
	option input		REJECT
	option output		ACCEPT
	option forward		REJECT
# Uncomment this line to disable ipv6 rules
#	option disable_ipv6	1

config zone
	option name		lan
	list   network		'LAN'
	list   network          'LAN6'
	option input		ACCEPT
	option output		ACCEPT
	option forward		REJECT

config zone
	option name		lan_secure
	list   network          'LAN_SECURE'
	option input            ACCEPT
	option output           ACCEPT
	option forward		REJECT

config zone
	option name             lan_iot
	list   network          'LAN_IOT'
	option input            ACCEPT
	option output           ACCEPT
	option forward          REJECT

config zone
	option name             lan_guest
	list network            'LAN_GUEST'
	option input            ACCEPT
	option output           ACCEPT
	option forward          REJECT

RFC1918 addresses do not need to be redacted... and in fact, when they are, it makes it more likely that we'll miss something. Would you mind revising your post such that these are not redacted

Router:

# cat /etc/config/network 
config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix '<ULA_PREFIX1>/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1:t'
	list ports 'lan2:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'lan1:t'
	list ports 'lan2:t'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option netmask '255.255.255.0'
	option ip6assign '64'
	list ip6class 'wan6'
	option ip6weight '0'
	option ipaddr '192.168.92.1'

config device
	option name 'wan'
	option macaddr '<WAN_MAC>'

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'

config device
	option name 'br-lan-secure'
	option type 'bridge'
	list ports 'br-lan.3'

config interface 'LAN_SECURE'
	option device 'br-lan-secure'
	option proto 'static'
	option ipaddr '192.168.91.1'
	option netmask '255.255.255.0'
	option ip6weight '3'
	option ip6assign '64'
	list ip6class 'WAN6SECURE'

config device
	option type 'bridge'
	option name 'br-lan-iot'
	option bridge_empty '1'
	list ports 'br-lan.2'

config interface 'LAN_IOT'
	option proto 'static'
	option device 'br-lan-iot'
	option ipaddr '192.168.89.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6weight '1'

config device
	option type 'bridge'
	option name 'br-lan-guest'
	option bridge_empty '1'
	list ports 'br-lan.1'

config interface 'LAN_GUEST'
	option proto 'static'
	option device 'br-lan-guest'
	option ipaddr '192.168.90.1'
	option netmask '255.255.255.0'
	option ip6assign '64'
	option ip6weight '2'
	list ip6class 'WAN6GUEST'

config device
	option type 'macvlan'
	option ifname 'wan'
	option mode 'vepa'
	option name 'wan-secure'
	option macaddr '<WAN_SECURE_MAC>'

config device
	option type 'macvlan'
	option ifname 'wan'
	option mode 'vepa'
	option name 'wan-guest'
	option macaddr '<WAN_GUEST_MAC>'

config interface 'WAN6SECURE'
	option proto 'dhcpv6'
	option device 'wan-secure'
	option reqaddress 'try'
	option reqprefix 'auto'

config interface 'WAN6GUEST'
	option proto 'dhcpv6'
	option device 'wan-guest'
	option reqaddress 'try'
	option reqprefix 'auto'

config interface 'WAN6IOT'
	option proto 'dhcpv6'
	option device 'wan-iot'
	option reqaddress 'try'
	option reqprefix 'auto'

config device
	option type 'macvlan'
	option ifname 'br-lan'
	option mode 'vepa'
	option name 'wan-iot'
	option macaddr '<WAN_IOT_MAC>'
# cat /etc/config/wireless 

config wifi-device 'radio0'
	option type 'mac80211'
	option hwmode '11a'
	option path 'soc/soc:pcie/pci0000:00/0000:00:01.0/0000:01:00.0'
	option country 'US'
	option cell_density '0'
	option htmode 'VHT80'
	option channel '36'

config wifi-device 'radio1'
	option type 'mac80211'
	option hwmode '11g'
	option path 'soc/soc:pcie/pci0000:00/0000:00:02.0/0000:02:00.0'
	option htmode 'HT20'
	option country 'US'
	option cell_density '0'
	option channel '6'
	option disabled '1'

config wifi-device 'radio2'
	option type 'mac80211'
	option channel '34'
	option hwmode '11a'
	option path 'platform/soc/soc:internal-regs/f10d8000.sdhci/mmc_host/mmc0/mmc0:0001/mmc0:0001:1'
	option htmode 'VHT80'
	option disabled '1'

config wifi-iface 'wifinet0'
	option device 'radio1'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option key '<IOT_PASSWORD>'
	option disassoc_low_ack '0'
	option macaddr '<ROUTER_WLAN2_MAC>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan1-iot'
	option disabled '1'

config wifi-iface 'wifinet4'
	option device 'radio0'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option key '<SECURE_PASSWORD>'
	option network 'LAN_SECURE'
	option macaddr '<ROUTER_WLAN5_MAC>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-secure'

config wifi-iface 'wifinet5'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT2_SSID>'
	option key '<IOT2_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option disabled '1'

config wifi-iface 'wifinet6'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option disassoc_low_ack '0'
	option key '<IOT_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-iot'

config wifi-iface 'wifinet7'
	option device 'radio0'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option key '<GUEST_PASSWORD>'
	option network 'LAN_GUEST'
	option wpa_disable_eapol_key_retries '1'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ifname 'wlan0-guest'

config wifi-iface 'wifinet8'
	option device 'radio1'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option ifname 'wlan1-guest'
	option key '<GUEST_PASSWORD>'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_GUEST'
	option disabled '1'

config wifi-iface 'wifinet9'
	option device 'radio1'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option encryption 'psk2'
	option key '<SECURE_PASSWORD>'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_SECURE'
	option ifname 'wlan1-secure'
	option disabled '1'
# cat /etc/config/dhcp 

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	list address '/<HOST1_NAME>.local/192.168.89.<HOST1_IP>'
	list address '/<HOST2_NAME>.local/192.168.89.<HOST2_IP>'
	list server '1.1.1.1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'LAN_SECURE'
	option interface 'LAN_SECURE'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option force '1'
	option dhcpv6 'server'
	option ra 'server'
	list ra_flags 'managed-config'

config dhcp 'LAN_GUEST'
	option interface 'LAN_GUEST'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option force '1'
	option ra 'server'
	option dhcpv6 'server'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config host
	option name '<HOST1_NAME>'
	option dns '1'
	option mac '<HOST1_MAC>'
	option ip '192.168.89.<HOST1_IP>'

config host
	option name '<HOST2_NAME>'
	option dns '1'
	option mac '<HOST2_MAC>'
	option ip '192.168.89.<HOST2_IP>'

config dhcp 'LAN_IOT'
	option interface 'LAN_IOT'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option dhcpv6 'server'
	list ra_flags 'other-config'

config dhcp 'LAN'
	option interface 'LAN'
	option start '100'
	option limit '150'
	option leasetime '12h'

config host
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '192.168.92.2'
	option name 'roaming1'

config host
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '192.168.91.2'
	option name 'roaming1'

config host
	option name 'roaming1'
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '192.168.90.2'

config host
	option name 'roaming1'
	option dns '1'
	option mac '<ROAMING1_MAC>'
	option ip '192.168.89.2'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option ra_slaac '0'
	option dhcpv6 'server'

config host
	option dns '1'
	option ip '192.168.89.3'
	option name 'roaming2'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming2'
	option dns '1'
	option ip '192.168.90.3'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming3'
	option dns '1'
	option ip '192.168.91.3'
	option mac '<ROAMING2_MAC>'

config host
	option name 'roaming3'
	option dns '1'
	option ip '192.168.92.3'
	option mac '<ROAMING2_MAC>'
# cat /etc/config/firewall 

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'

config zone
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	option name 'lan_iot'
	list network 'LAN_IOT'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'WAN6SECURE'
	list network 'WAN6GUEST'
	list network 'WAN6IOT'

config forwarding
	option dest 'wan'
	option src 'lan_iot'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option src_ip 'fc00::/6'
	option dest_ip 'fc00::/6'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-IPSec-ESP'
	option src 'wan'
	option proto 'esp'
	option target 'ACCEPT'
	option dest 'lan_iot'

config rule
	option name 'Allow-ISAKMP'
	option src 'wan'
	option dest_port '500'
	option proto 'udp'
	option target 'ACCEPT'
	option dest 'lan_iot'

config rule
	option name 'Support-UDP-Traceroute'
	option src 'wan'
	option dest_port '33434:33689'
	option proto 'udp'
	option family 'ipv4'
	option target 'REJECT'
	option enabled 'false'

config include
	option path '/etc/firewall.user'

config zone
	option name 'lan_secure'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'LAN_SECURE'

config zone
	option name 'lan_guest'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'LAN_GUEST'

config forwarding
	option src 'lan_secure'
	option dest 'wan'

config forwarding
	option src 'lan_guest'
	option dest 'wan'

config forwarding
	option src 'lan_secure'
	option dest 'lan_iot'

config rule
	option name 'Allow-IOT-mDNS'
	list proto 'udp'
	list src_ip '224.0.0.251'
	option src_port '5353'
	list dest_ip '224.0.0.251'
	option dest_port '5353'
	option target 'ACCEPT'
	option family 'ipv4'
	option src 'lan_iot'

config rule
	option name 'Allow-IOT-mDNS6'
	option family 'ipv6'
	list proto 'udp'
	list dest_ip 'ff02::fb'
	option dest_port '5353'
	option target 'ACCEPT'
	option src 'lan_iot'

config zone
	option input 'ACCEPT'
	option output 'ACCEPT'
	option name 'lan'
	option forward 'ACCEPT'
	list network 'LAN'

config forwarding
	option src 'lan'
	option dest 'wan'

Roaming:

# cat /etc/config/network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option ula_prefix '<ULA_PREFIX2>/48'

config device
	option name 'br-lan'
	option type 'bridge'
	option macaddr '<ROAMING1_MAC>'
	option empty_bridge '1'
	list ports 'wan'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'wan:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'wan:t'
	list ports 'lan1:u'
	list ports 'lan2:u'
	list ports 'lan3:u'
	list ports 'lan4:u'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'wan:t'

config interface 'LAN'
	option device 'br-lan'
	option proto 'dhcp'

config device
	option name 'br-lan-guest'
	option type 'bridge'
	option empty_bridge 1
	list ports 'br-lan.1'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_GUEST'
	option device 'br-lan-guest'
	option proto 'dhcp'

config device
	option name 'br-lan-iot'
	option type 'bridge'
	option empty_bridge 1
	list ports 'br-lan.2'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_IOT'
	option device 'br-lan-iot'
	option proto 'dhcp'

config device
	option name 'br-lan-secure'
	option type 'bridge'
	list ports 'br-lan.3'
	option macaddr '<ROAMING1_MAC>'

config interface 'LAN_SECURE'
	option device 'br-lan-secure'
	option proto 'dhcp'
# cat /etc/config/wireless 

# 2.4 GHz WiFi is disabled on roaming2
config wifi-device 'radio0'
	option type 'mac80211'
	option path 'platform/soc/18000000.wmac'
	option band '2g'
	option htmode 'HE20'
	option channel '6'
	option cell_density '0'

config wifi-device 'radio1'
	option type 'mac80211'
	option path 'platform/soc/18000000.wmac+1'
	option band '5g'
	option cell_density '0'
	option htmode 'HE80'
	option channel '36'

config wifi-iface 'wifinet0'
	option device 'radio1'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option key '<IOT_PASSWORD>'
	option disassoc_low_ack '0'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'

config wifi-iface 'wifinet4'
	option device 'radio0'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option key '<SECURE_PASSWORD>'
	option network 'LAN_SECURE'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'

config wifi-iface 'wifinet6'
	option device 'radio0'
	option mode 'ap'
	option ssid '<IOT_SSID>'
	option disassoc_low_ack '0'
	option key '<IOT_PASSWORD>'
	option encryption 'psk2'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_IOT'

config wifi-iface 'wifinet7'
	option device 'radio0'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option key '<GUEST_PASSWORD>'
	option network 'LAN_GUEST'
	option wpa_disable_eapol_key_retries '1'

config wifi-iface 'wifinet8'
	option device 'radio1'
	option mode 'ap'
	option ssid '<GUEST_SSID>'
	option encryption 'psk2'
	option ifname 'wlan1-guest'
	option key '<GUEST_PASSWORD>'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_GUEST'

config wifi-iface 'wifinet9'
	option device 'radio1'
	option mode 'ap'
	option ssid '<SECURE_SSID>'
	option encryption 'psk2'
	option key '<SECURE_PASSWORD>'
	option wpa_disable_eapol_key_retries '1'
	option network 'LAN_SECURE'
# cat /etc/config/dhcp 

config dnsmasq
	option domainneeded '1'
	option boguspriv '1'
	option filterwin2k '0'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option nonegcache '0'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option nonwildcard '1'
	option localservice '1'
	option ednspacket_max '1232'
	option filter_aaaa '0'
	option filter_a '0'

config dhcp 'lan'
	option interface 'lan'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option dhcpv4 'server'
	option dhcpv6 'server'
	option ra 'server'
	option ra_slaac '1'
	list ra_flags 'managed-config'
	list ra_flags 'other-config'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'
# cat /etc/config/firewall 
config defaults
	option syn_flood	1
	option input		REJECT
	option output		ACCEPT
	option forward		REJECT
# Uncomment this line to disable ipv6 rules
#	option disable_ipv6	1

config zone
	option name		lan
	list   network		'LAN'
	list   network          'LAN6'
	option input		ACCEPT
	option output		ACCEPT
	option forward		REJECT

config zone
	option name		lan_secure
	list   network          'LAN_SECURE'
	option input            ACCEPT
	option output           ACCEPT
	option forward		REJECT

config zone
	option name             lan_iot
	list   network          'LAN_IOT'
	option input            ACCEPT
	option output           ACCEPT
	option forward          REJECT

config zone
	option name             lan_guest
	list network            'LAN_GUEST'
	option input            ACCEPT
	option output           ACCEPT
	option forward          REJECT

I think I see the problem.... it looks like you're 'nesting' bridge interfaces.

for example...

The middle stanza is invalid. That should be deleted. Then, your LAN_SECURE network will use device br-lan.3.

You've got a lot going on in those files, so there's probably a lot of cleanup work to be done. See if you can make some progress based on my comment above, but consider that it might be best to start from scratch and then add just a single network at a time (verifying that it is working before moving onto the next).

They do work until they don't so checking "if it works" is not a quick check.

Well, its hard to know why you might have intermittent situations where it isn't working, but there are certainly problems in the config -- it's hard to know if that's the actual culprit or just a red herring.

Starting over may be the best option... you'll go from something that doesn't have any VLANs at all to something that is working for one additional network and validated to be the correct implementation by the forum here. Then you can create the other networks using the same recipe. Finally, you can do the long term testing after that to determine if the network is more reliable.

As an aside, there are other situations where flaky hardware can cause similar symptoms... but it's given that you've got known issues with the OpenWrt config, I'd start there. If the problem manifests after you've fixed OpenWrt's config, we can look at other possible causes.

Ok. The problem is that multiple networks is how I started. Than I got need for multiple routers after I moved and I didn't have range in whole home.

I'm not sure I understand... do your other routers operate as dumb APs, or are they actively routing? Can you draw a topology diagram?

Hmm. I'm not sure if a diagram will clear up.

  • I had a single router with 3 networks for years. Several of my systems depend on the configuration. The configuration was generated by LUCI so yeah, it's a mess.
  • When I moved to new place I noticed that I don't have coverage in lot of places as the ISP network comes in a corner.
  • I added 2 new routers (roaming1 and roaming2) and tries to get the thing to work. I tried to stitch together a system where they work.
  • My plan was for roaming1 and roaming2 to expose WiFis and use CAT6 for interconnection.
  • Unfortunately the VLAN/bridge setup in OpenWRT is hard, at least to someone who is not a network engineer. A lot of tutorials are either having a single router with multiple networks or multiple router with single network. And wiki describes an outdated interface so is no help either.
  • After I tried to do it, it worked - mostly. I think the issue is when I roam between APs something 'cashes' where the MAC address is on return path.

Hopefully it will be fixed by removing one layer of bridging but given that networks are copy'n'paste of each other essentially I'm not sure if doing it one by one will help debugging (but will create churn for me to change settings in devices).