Linksys WRT1900ACv1 Vlan routing issue

For context I have been running the same configuration for around a year now and have not had a problem. I just updated my router to OpenWrt 23.05.3 and have noticed some strange traffic dropouts on certain vlans. I have 3 Vlans one for general computers one for IOT devices and one for Camera devices. On the general computers vlan I have full forwarding to the wan with no issues. On the Camera and IOT vlan I have forwarding to wan disabled. I enabled the forwarding on a device directly though firewall traffic rules. So generally devices that join that network can not access the internet nor anything else. They only get DHCP and can talk intervlan but nothing else. There are a few devices on each vlan that I want to have internet access to. I have setup firewall rules to allow all forwarding from an IP address to the wan. That allows me to send pings and do dns lookup no problems, however when I do a curl command it always fails. Doing curl command while wireshark is running I can see the TCP handshake send the syn and get a syn ack. Once I get a syn ack the device then sends a bunch of tcp retransmissions. I have tried a number of different rules to try and get it to work. This same setup worked on 22.03.2 but breaks now on 23.05.3. The rules work when the traffic is from the wireless network on the same firewall rule but will not work with wired traffic from a vlan. Is there any suggestions or solutions that will help with the problem? I am happy to give more information or make changes if it will help lead to a solution.

Please connect to your OpenWrt device using ssh and copy the output of the following commands and post it here using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have:

ubus call system board
cat /etc/config/network
cat /etc/config/wireless
cat /etc/config/dhcp
cat /etc/config/firewall

Current config

ubus call system board:

{
	"kernel": "5.15.150",
	"hostname": "home-ingress",
	"system": "ARMv7 Processor rev 2 (v7l)",
	"model": "Linksys WRT1900AC v1",
	"board_name": "linksys,wrt1900ac-v1",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "23.05.3",
		"revision": "r23809-234f1a2efa",
		"target": "mvebu/cortexa9",
		"description": "OpenWrt 23.05.3 r23809-234f1a2efa"
	}
}

cat /etc/config/network:

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option packet_steering '1'
	option ula_prefix 'fd64:9ad:12f1::/48'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'

config device
	option name 'lan1'
	option macaddr <redacted>

config device
	option name 'lan2'
	option macaddr <redacted>

config device
	option name 'lan3'
	option macaddr <redacted>

config device
	option name 'lan4'
	option macaddr <redacted>

config device
	option name 'wan'
	option macaddr <redacted>

config interface 'wan'
	option device 'wan'
	option proto 'dhcp'
	option delegate '0'

config bridge-vlan
	option device 'br-lan'
	option vlan '6'
	list ports 'lan1:t'
	list ports 'lan3:u*'
	list ports 'lan4:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1:u*'
	list ports 'lan2:u*'
	list ports 'lan4:u*'

config interface 'IOT_VLAN'
	option proto 'static'
	option device 'br-lan.6'
	option netmask '255.255.255.0'
	option delegate '0'
	option ipaddr '192.168.6.1'

config interface 'ACCESS_VLAN'
	option proto 'static'
	option device 'br-lan.10'
	option netmask '255.255.255.0'
	option type 'bridge'
	option ipaddr '192.168.10.1'
	option ip6assign '64'
	option ip6hint '10'
	list ip6class 'WANv6'
	option ip6weight '10'
	option delegate '0'

config device
	option name 'br-lan.10'
	option type '8021q'
	option ifname 'br-lan'
	option vid '10'

config device
	option name 'br-lan.6'
	option type '8021q'
	option ifname 'br-lan'
	option vid '6'

config interface 'Wireguard'
        <redacted>

config wireguard_Wireguard
	<redacted>

config wireguard_Wireguard
	<redacted>

config wireguard_Wireguard
	<redacted>

config wireguard_Wireguard
	<redacted>

config device
	option type '8021q'
	option ifname 'br-lan'
	option vid '4'
	option name 'br-lan.4'

config interface 'CAM_VLAN'
	option proto 'static'
	option device 'br-lan.4'
	list ipaddr '192.168.4.1/24'
	option ip6assign '64'
	list ip6class 'WANv6'
	option ip6hint '4'
	option delegate '0'

config bridge-vlan
	option device 'br-lan'
	option vlan '4'
	list ports 'lan1:t'
	list ports 'lan4:t'

config wireguard_Wireguard
	<redacted>

config interface 'WANv6'
	option proto 'dhcpv6'
	option device 'wan'
	option reqaddress 'try'
	option reqprefix 'auto'

cat /etc/config/wireless:

config wifi-device 'radio0'
	option type 'mac80211'
	option path 'soc/soc:pcie@82000000/pci0000:00/0000:00:02.0/0000:02:00.0'
	option band '2g'
	option country 'US'
	option channel 'auto'
	option cell_density '0'
	option htmode 'HT40'

config wifi-device 'radio1'
	option type 'mac80211'
	option path 'soc/soc:pcie@82000000/pci0000:00/0000:00:03.0/0000:03:00.0'
	option band '5g'
	option htmode 'VHT80'
	option country 'US'
	option cell_density '0'
	option channel 'auto'

config wifi-iface 'wifinet0'
	option device 'radio0'
	option mode 'ap'
	option key <redacted>
	option network 'ACCESS_VLAN'
	option ssid <redacted>
	option macaddr <redacted>
	option encryption 'psk2'

config wifi-iface 'wifinet1'
	option device 'radio0'
	option mode 'ap'
	option encryption 'psk2'
	option key <redacted>
	option ssid <redacted>
	option macaddr <redacted>
	option isolate '1'
	option network 'ACCESS_VLAN'

config wifi-iface 'wifinet2'
	option device 'radio0'
	option mode 'ap'
	option encryption 'psk2'
	option key <redacted>
	option network 'IOT_VLAN'
	option ssid <redacted>
	option macaddr <redacted>

config wifi-iface 'wifinet3'
	option device 'radio1'
	option mode 'ap'
	option key <redacted>
	option network 'ACCESS_VLAN'
	option ssid <redacted>
	option encryption 'psk2'

config wifi-iface 'wifinet4'
	option device 'radio0'
	option mode 'ap'
	option encryption 'psk2'
	option key <redacted>
	option network 'CAM_VLAN'
	option isolate '1'
	option ssid <redacted>
	option macaddr <redacted>

cat /etc/config/dhcp:

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option rebind_protection '1'
	option rebind_localhost '1'
	option local '/lan/'
	option authoritative '1'
	option readethers '1'
	option leasefile '/tmp/dhcp.leases'
	option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
	option localservice '1'
	option ednspacket_max '1232'
	option domain <redacted>
	list notinterface 'wan'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'IOT_VLAN'
	option interface 'IOT_VLAN'
	option start '100'
	option limit '150'
	option leasetime '12h'
	list dhcp_option '6,9.9.9.11,149.112.112.11'

config dhcp 'ACCESS_VLAN'
	option interface 'ACCESS_VLAN'
	option start '100'
	option leasetime '12h'
	option limit '50'
	list dhcp_option '6,192.168.10.7'
	option ra 'server'
	option dhcpv6 'server'
	list dns 'fe80::882d:74ff:fe9b:475d'
	list ra_flags 'other-config'

<redacted all static leases>

cat /etc/config/firewall:

config defaults
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option synflood_protect '1'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option name 'Access'
	list network 'ACCESS_VLAN'

config zone
	option name 'Wireguard'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'Wireguard'

config zone
	option name 'Camera'
	option output 'ACCEPT'
	option forward 'REJECT'
	option input 'REJECT'
	list network 'CAM_VLAN'
	option family 'ipv4'

config zone
	option name 'IOT'
	option forward 'REJECT'
	option input 'REJECT'
	option output 'ACCEPT'
	list network 'IOT_VLAN'

config zone
	option name 'wan'
	option input 'REJECT'
	option output 'ACCEPT'
	option forward 'REJECT'
	option masq '1'
	option mtu_fix '1'
	list network 'wan'
	list network 'WANv6'

config forwarding
	option dest 'wan'
	option src 'Access'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option target 'ACCEPT'
	option family 'ipv4'
	option dest_port '68'

config rule
	option src 'wan'
	option proto 'icmp'
	option target 'ACCEPT'
	list icmp_type 'echo-request'
	option family 'ipv4'
	option name 'Allow-ICMP'

config rule
	option name 'Allow-DHCPv6-Renew'
	option family 'ipv6'
	list proto 'udp'
	option src 'wan'
	option dest_port '546'
	option target 'ACCEPT'
	list src_ip 'fc00::/6'
	list dest_ip 'fc00::/6'

config rule
	option name 'Allow-MLD'
	option family 'ipv6'
	list proto 'icmp'
	option src 'wan'
	list src_ip 'fe80::/10'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMP6'
	option family 'ipv6'
	list proto 'icmp'
	list icmp_type 'bad-header'
	list icmp_type 'destination-unreachable'
	list icmp_type 'echo-reply'
	list icmp_type 'echo-request'
	list icmp_type 'neighbour-advertisement'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'packet-too-big'
	list icmp_type 'router-advertisement'
	list icmp_type 'router-solicitation'
	list icmp_type 'time-exceeded'
	list icmp_type 'unknown-header-type'
	option src 'wan'
	option target 'ACCEPT'
	option limit '1000/second'

config rule
	option name 'Allow-ICMPv6-Forward'
	list proto 'icmp'
	option src 'wan'
	option dest '*'
	option target 'ACCEPT'
	option family 'ipv6'
	list icmp_type 'bad-header'
	list icmp_type 'destination-unreachable'
	list icmp_type 'echo-reply'
	list icmp_type 'echo-request'
	list icmp_type 'neighbour-advertisement'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'packet-too-big'
	list icmp_type 'router-advertisement'
	list icmp_type 'router-solicitation'
	list icmp_type 'time-exceeded'
	list icmp_type 'unknown-header-type'
	option limit '100/second'

config rule
	option name 'DHCP-IOT'
	list proto 'udp'
	option src 'IOT'
	option dest_port '67 68'
	option target 'ACCEPT'

config rule
	list proto 'udp'
	option src 'Camera'
	option target 'ACCEPT'
	option name 'DHCP-CAM'
	option dest_port '67 68 546 547'

config rule
	option name 'Wireguard - HA'
	option family 'ipv4'
	list proto 'tcp'
	option src 'Wireguard'
	option dest 'IOT'
	list dest_ip '192.168.6.25'
	option dest_port '8123 443 80'
	option target 'ACCEPT'

config rule
	option name 'TV-Jellyfin'
	list proto 'tcp'
	option src 'IOT'
	list src_ip '192.168.6.89'
	list src_ip '192.168.6.90'
	option dest 'Access'
	option target 'ACCEPT'
	option family 'ipv4'
	list dest_ip '192.168.10.50'
	list dest_ip '192.168.10.65'

config rule
	option name 'TV-Out'
	option family 'ipv4'
	list proto 'all'
	option src 'IOT'
	list src_ip '192.168.6.90'
	list src_ip '192.168.6.89'
	option dest 'wan'
	option target 'ACCEPT'

config include
	option path '/etc/firewall.user'

config forwarding
	option dest 'IOT'
	option src 'Access'

config forwarding
	option src 'Access'
	option dest 'Wireguard'

config forwarding
	option src 'Wireguard'
	option dest 'Access'

config forwarding
	option src 'Wireguard'
	option dest 'wan'

config forwarding
	option src 'Wireguard'
	option dest 'IOT'

config rule
	option name 'Test'
	option family 'ipv4'
	option src 'IOT'
	option dest 'wan'
	option target 'ACCEPT'
	list proto 'all'
	list src_ip '192.168.6.25'
	list src_ip '192.168.6.116'
	list src_ip '192.168.6.141'

Remove the bridge line from below:

Delete these:

And this:

Restart and try again.

Removed the devices and removed the bridge type from the accessvlan interface with no change to results. The devices are gone but I still cant do a http curl from the IOT vlan from ip address 192.168.6.25

From 192.168.6.25 to what address/subnet/vlan?

192.168.6.25 can properly curl to any internal subnet (with the addtion of another firewall rule to allow it to reach the other subnet) and can properly do dns lookup and pings externally but can not complete a curl request to the wan (testing by curl google.com).

What happens if you do this:

curl 64.226.122.113

Same issue. Curl on port 80 seems to go through eventually (about 30 seconds to 1 minute to get a response and most of the time an empty response) and on port 443 it errors out with curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to google.com:443

Check the time on the affected host. Incorrect time can cause issues with the encryption.

Time is correct. This affects all hosts on the wired network. And the encryption is only a partial to the whole problem. The standard unencrypted transfer is also not correct. From Vlan 10 the time to respond is less than a second compared to almost a minute on vlan 6. Plus this only happens in OpenWrt 23.05 and works fine in OpenWrt 22.03

What happens if you allow a forward from the IoT zone > wan zone (just as a temporary test).

Doing an allow for forwarding IoT zone to the Wan zone does not fix the problem.

Have you tried another host (just to rule out the specific one from which you are testing)?

Yes. I have tried multiple hosts in different locations routing through different mediums. The only method that works is through WiFi. Any wired method fails to work.

I'm not seeing any reason why wired would be treated any differently than wireless devices.

When you are testing wired and wireless on the IoT network, can you confirm that they are indeed on the same subnet (please double check, just to make sure we are looking at the right things)?

Do you have any additional APs on your network?

They are all on the same subnet. Both wired and wireless on the 192.168.6.0/24 subnet. There is no additional ap’s on the network.

Some more information after doing a tcpdump and some analysis it seems traffic on the IoT vlan is just slow to get a response from the external sever with seemingly no difference in traffic patterns. I ran a curl to google.com on both the IOT and ACCESS vlans which both get a response back but when coming from the ACCESS Vlan it is around 0.01 second compared to about 1 minute 50 seconds. And it seems that the length of time it takes to get data breaks the SSL ablitiy.