Suggestions to troubleshoot IPv6 routing to default gateway

I am trying to complete the "final" step in a project I've been working on, and have setup a raspberry Pi router in my truck that connects to the internet over starlink, via a a wireguard tunnel. I have had this working flawlessly as a configuration on my home router, as a point in reference.
The home router connects to the new rpi router over a wireless WDS bridge

This weekend, I cut out the relevent config from my home router and moved them 1:1 to the RPi router. From the perspective of the RPi router, everything works exactly as expected, including ipv6 and ipv4 routing over the wireguard tunnel to the internet.

The issue I'm trying to solve is only ipv6 routing from my home router doesn't work. I can see the traffic arrive to the interface of the rpi, but somewhere "inside" the router the traffic destined for the default route, the traffic just disappears. The firewall zone is set to allow all input and forward from the bridge network.
The part that is frustrating is that IPv6 traffic on the RPI router works, and I can connect to the RPI's interfaces/hosts from the home router, so it's just traffic that follows the default route that dies.

Hope this helps.

Thanks, but the issue is isolated from starlink. I route all internet traffic across a wireguard tunnel, which is there the ::/0 default route points to on both the home router and the rpi router. In other words, Starlink never sees any of the traffic I'm trying to route, beyond the tunnel traffic.

Something on the rpi router is dropping the ipv6 traffic after it is received on the wifi bridge, but like I said there is no firewall drop/reject rules in the way. I can still ping to ipv6 internet from the rpi router.

ip route get <dst_ip> from <src_ip_of_interface>

Have you verified the route?

ip route get fd00:f9a8:0:2::1 from fd00:f9a8:42::1
fd00:f9a8:0:2::1 from fd00:f9a8:42::1 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto 196 src fd00:f9a8:9a7e:300::2 metric 20 pref medium

I didn't check with that command specifically but here's the result, it indicates there's a route for the fd00:f9a8:0:2::1 (via default) for the home router subnet fd00:f9a8:0:42::/64

Just an idea:
Usually WireGuard has an ULA IPv6 address if you want internet access via the WG tunnel you have to NAT66 the WG IPv6 ULA out via the WAN6.

I do not know if that is something which is standard already implemented (and with nftables I cannot help either I am still very much an iptable guy )

1 Like

Could you make a diagram of the topology, just so it is clear?
Then run the following commands (copy-paste the whole block) and paste the output here, using the "Preformatted text </> " button:
grafik
Remember to redact passwords, MAC addresses and any public IP addresses you may have

ubus call system board; \
uci export network; uci export wireless; \
uci export dhcp; uci export firewall; \
ip -4 addr ; ip -4 ro li tab all ; ip -4 ru; \
ip -6 addr ; ip -6 ro li tab all ; ip -6 ru; \
ls -l  /etc/resolv.* /tmp/resolv.* /tmp/resolv.*/* ; head -n -0 /etc/resolv.* /tmp/resolv.* /tmp/resolv.*/*
1 Like

Here's a rough network map. vbond 0 on the home router is a direct link to the SDWAN VPS, this works for ipv6 connectivity when the RPI router link is off/disabled due to OSPF cost 1000

The RPI has vbond 0-3 wireguard interfaces, vbond0 links through starlink, vbond1 links through another cellular modem, vbond3 is currently unused.

Both the home router and the RPI have ::/0 default route set to use the 399::1 ip on the SDWAN VPS, this creates a recursive route over the lowest cost wireguard interface that's online. Routing is setup by frr/ospf

Making another post due to char limit....and the firewall rules to prevent WAN output is intentional, only the wireguard fw mark is permitted to hit the internet

Client Vlan fd00:f9a8:0:42::/64
                    ||
             Home Router------cell-modem1---wireguard------------SDWAN, default gateway fd00:f9a8:9a7e:399::1---------internet
                  ||                                                 |
              WiFi Bridge                                            |
                  ||                                                 |
             RPI router--------cell-modem2-----wireguard--------------
                      |                                              |
                  Starlink---------wireguard--------------------------

cmd output of home router

{
	"kernel": "5.10.176",
	"hostname": "npancwangw01.domain.net",
	"system": "ARMv8 Processor rev 4",
	"model": "Linksys E8450 (UBI)",
	"board_name": "linksys,e8450-ubi",
	"rootfs_type": "squashfs",
	"release": {
		"distribution": "OpenWrt",
		"version": "22.03.4",
		"revision": "r20123-38ccc47687",
		"target": "mediatek/mt7622",
		"description": "OpenWrt 22.03.4 r20123-38ccc47687"
	}
}
package network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option packet_steering '1'

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan1'
	list ports 'lan2'
	list ports 'lan3'

config interface 'wan'
	option device 'wan'
	option proto 'static'
	option delegate '0'
	list ipaddr '10.255.0.66/30'
	list ip6addr 'fd00:f9a8:9a7e:aedc::1/64'
	list ip6addr 'fe80::fc43:9abd:5c33:5933/64'

config bridge-vlan
	option device 'br-lan'
	option vlan '1'
	list ports 'lan1'

config bridge-vlan
	option device 'br-lan'
	option vlan '2'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3:u*'

config bridge-vlan
	option device 'br-lan'
	option vlan '3'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '10'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3:t'

config bridge-vlan
	option device 'br-lan'
	option vlan '11'
	list ports 'lan1:t'
	list ports 'lan2:t'
	list ports 'lan3:t'

config interface 'admin'
	option proto 'static'
	list ipaddr '10.0.18.1/24'
	option device 'br-admin'
	option defaultroute '0'
	option delegate '0'
	option ip6weight '100'
	list ip6addr 'FD00:F9A8:0:42::1/64'

config interface 'devices'
	option proto 'static'
	list ipaddr '10.0.19.1/24'
	option device 'br-devices'
	option delegate '0'
	option defaultroute '0'
	list ip6addr 'FD00:F9A8:0:43::1/64'

config interface 'dummy'
	option proto 'static'
	option device 'dummy0'
	option delegate '0'
	option defaultroute '0'
	list ipaddr '10.23.32.2/32'
	list ip6addr 'fd00:f9a8:9a7e:399::3/128'

config interface 'domain'
	option proto 'static'
	option delegate '0'
	option defaultroute '0'
	option device 'br-domain'
	list ipaddr '10.0.53.1/29'
	list ip6addr 'fd00:f9a8:53:1::1/64'

config interface 'guest'
	option proto 'static'
	option device 'br-guest'
	option delegate '0'
	option defaultroute '0'
	list ipaddr '10.0.23.1/24'
	list ip6addr 'FD00:F9A8:0:46::1/64'

config interface 'resident'
	option proto 'static'
	option device 'br-resident'
	option delegate '0'
	option defaultroute '0'
	list ipaddr '10.0.22.1/24'
	list ip6addr 'fd00:f9a8:0:45::1/64'

config interface 'ovpn'
	option proto 'none'
	option defaultroute '0'
	option peerdns '0'
	option delegate '0'
	option device 'tun0'

config device
	option type 'bridge'
	option name 'br-admin'
	list ports 'br-lan.2'
	list ports 'wan.2'

config device
	option type 'bridge'
	option name 'br-devices'
	list ports 'br-lan.3'

config device
	option type 'bridge'
	option name 'br-resident'
	list ports 'br-lan.10'

config device
	option type 'bridge'
	option name 'br-guest'
	list ports 'br-lan.11'

config interface 'wg0'
	option proto 'wireguard'
	option nohostroute '1'
	option defaultroute '0'
	option peerdns '0'
	option ip4table '54'
	option ip6table '54'
	option delegate '0'
	option fwmark '0x34dd'
	list addresses '10.33.23.2/30'
	list addresses 'fd00:f9a8:9a7e:300::2/64'
	list addresses 'fe80::6331:aabc:dd31:2e9e/64'
	option mtu '1340'
	option private_key ''

config wireguard_wg0
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option endpoint_host ''
	option public_key ''
	option preshared_key ''
	option description 'nvrnsdwangw01-wwan'
	option endpoint_port '9470'

config rule
	option lookup '55'
	option mark '0x34dd'

config rule
	option lookup '55'
	option dest '192.168.5.1/32'

config interface 'wwan'
	option device 'lan4'
	option proto 'dhcp'
	option ip4table '55'
	option ip6table '55'
	option delegate '0'
	option peerdns '0'

config interface 'wwan6'
	option proto 'dhcpv6'
	option reqaddress 'try'
	option reqprefix 'auto'
	option peerdns '0'
	option ip4table '55'
	option ip6table '55'
	option delegate '0'
	option device '@wwan'

config device
	option type 'bridge'
	option name 'br-domain'
	option bridge_empty '1'

package wireless

config wifi-device 'radio0'
	option type 'mac80211'
	option path 'platform/18000000.wmac'
	option band '2g'
	option htmode 'HT20'
	option country 'US'
	option cell_density '0'
	option channel 'auto'

config wifi-device 'radio1'
	option type 'mac80211'
	option path '1a143000.pcie/pci0000:00/0000:00:00.0/0000:01:00.0'
	option band '5g'
	option country 'US'
	option cell_density '0'
	option he_su_beamformee '1'
	option he_bss_color '8'
	option htmode 'HE80'
	option channel 'auto'
	option channels '100-165'

config wifi-iface 'wifinet0'
	option device 'radio0'
	option mode 'ap'
	option ssid 'domain Devices'
	option key ''
	option network 'devices'
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option wps_pushbutton '1'
	option mobility_domain '4f53'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'
	option encryption 'psk-mixed+ccmp'

config wifi-iface 'wifinet1'
	option device 'radio0'
	option mode 'ap'
	option ssid 'domain Resident'
	option key ''
	option ieee80211w '2'
	option network 'resident'
	option ieee80211r '1'
	option mobility_domain '4f50'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option encryption 'sae-mixed'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'

config wifi-iface 'wifinet2'
	option device 'radio0'
	option mode 'ap'
	option ssid 'domain Visitor'
	option network 'guest'
	option key ''
	option ieee80211r '1'
	option mobility_domain '4f51'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option ieee80211w '1'
	option encryption 'sae-mixed'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'

config wifi-iface 'wifinet3'
	option device 'radio1'
	option mode 'ap'
	option ssid 'domain Resident'
	option encryption 'sae-mixed'
	option key ''
	option ieee80211w '2'
	option network 'resident'
	option ieee80211r '1'
	option mobility_domain '4f50'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'

config wifi-iface 'wifinet4'
	option device 'radio1'
	option mode 'ap'
	option ssid 'domain Visitor'
	option encryption 'sae-mixed'
	option network 'guest'
	option key ''
	option ieee80211r '1'
	option mobility_domain '4f51'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'

config wifi-iface 'wifinet5'
	option device 'radio1'
	option mode 'ap'
	option ssid 'domain Devices'
	option key ''
	option ieee80211r '1'
	option ft_over_ds '0'
	option ft_psk_generate_local '1'
	option network 'devices'
	option time_advertisement '2'
	option time_zone 'AKST9AKDT,M3.2.0,M11.1.0'
	option wnm_sleep_mode '1'
	option bss_transition '1'
	option ieee80211k '1'
	option mobility_domain '4f53'
	option encryption 'psk-mixed+ccmp'

package dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option local '/lan/'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option ednspacket_max '1232'
	option rebind_protection '0'
	option domain 'lan'
	option localservice '0'
	option confdir '/mnt/app/dnsmasq/'
	option cachesize '1000'
	option filterwin2k '1'
	option leasefile '/mnt/app/dhcp/dhcp.leases'
	option allservers '1'
	option dnsforwardmax '1000'
	list server '1.1.1.1'
	list server '1.0.0.1'
	list interface 'guest'
	list interface 'admin'
	list interface 'devices'
	list interface 'resident'
	list interface 'domain'
	list notinterface 'wan'
	list notinterface 'wwan'
	list notinterface 'vbond0'
	list notinterface 'vbond1'
	list notinterface 'vbond2'
	list address '/hassio.domain.net/10.0.19.27'
	list address '/dishy.starlink.com/192.168.100.1'
	list address '/vpn.domain.net/10.0.22.1'
	list address '/vpn.domain.net/10.0.23.1'
	option noresolv '1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	option start '100'
	option limit '150'
	option leasetime '12h'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'admin'
	option interface 'admin'
	option ra 'server'
	option ra_default '2'
	list domain 'domain.net'
	option start '100'
	option leasetime '12h'
	option limit '150'
	list dns 'fd00:f9a8:53:1::2'
	list dns 'fd00:f9a8:53:2::10'
	list dhcp_option '15,domain.net'
	list dhcp_option '6,10.0.53.2,10.0.53.10'
	list ra_flags 'other-config'
	option dhcpv6 'server'

config dhcp 'domain'
	option interface 'domain'
	option dynamicdhcp '0'
	option ignore '1'
	option leasetime '12h'
	option limit '150'
	option start '100'

config dhcp 'guest'
	option start '100'
	option leasetime '12h'
	option limit '150'
	option interface 'guest'
	option ra 'server'
	option ra_management '1'
	option ra_default '2'
	list domain 'lan'
	list dns 'fd00:f9a8:0:46::1'
	list ra_flags 'other-config'
	option dhcpv6 'server'

config dhcp 'resident'
	option interface 'resident'
	option ra 'server'
	list domain 'domain.net'
	option ra_default '2'
	option leasetime '12h'
	option start '100'
	option limit '150'
	list dns 'fd00:f9a8:53:1::2'
	list dns 'fd00:f9a8:53:2::10'
	list dhcp_option '15,domain.net'
	list dhcp_option '6,10.0.53.2,10.0.53.10'
	list ra_flags 'other-config'
	option dhcpv6 'server'

config dhcp 'devices'
	option interface 'devices'
	list domain 'domain.net'
	option ra_default '2'
	option start '100'
	option leasetime '12h'
	option limit '150'
	option ra 'server'
	option force '1'
	list dns 'fd00:f9a8:53:1::2'
	list dns 'fd00:f9a8:53:2::10'
	list dhcp_option '15,domain.net'
	list dhcp_option '6,10.0.53.2,10.0.53.10'
	list ra_flags 'other-config'
	option dhcpv6 'server'


package firewall

config defaults
	option output 'ACCEPT'
	option synflood_protect '1'
	option drop_invalid '1'
	option input 'DROP'
	option forward 'DROP'
	option flow_offloading '1'
	option flow_offloading_hw '1'

config zone
	option input 'DROP'
	option forward 'DROP'
	option name 'wan'
	option mtu_fix '1'
	option output 'DROP'
	list network 'wwan'
	list network 'wwan6'

config zone
	option name 'homebridge'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'wan'
	option input 'REJECT'

config zone
	option name 'sdwan_bond'
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'REJECT'
	option masq_allow_invalid '1'
	option mtu_fix '1'
	option log '1'
	list network 'dummy'
	list network 'wg0'

config zone
	option name 'ovpn'
	option output 'ACCEPT'
	option mtu_fix '1'
	option forward 'ACCEPT'
	option log '1'
	option input 'ACCEPT'
	list network 'ovpn'

config zone
	option name 'admin'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'admin'

config zone
	option name 'domain'
	option output 'ACCEPT'
	option mtu_fix '1'
	option forward 'ACCEPT'
	option input 'REJECT'
	list network 'domain'

config rule
	option target 'ACCEPT'
	list proto 'ospf'
	option src 'homebridge'
	option name 'OSPF Bridge'

config rule
	option name 'OSPF WG'
	list proto 'ospf'
	option src 'sdwan_bond'
	option target 'ACCEPT'

config rule
	option direction 'out'
	option name 'Block sdwan wwan'
	option dest 'wan'
	option device 'lan4'
	option target 'ACCEPT'
	option mark '0x34dd'
	list proto 'all'

config rule
	option name 'SDWAN HAProxy'
	list proto 'tcp'
	option target 'ACCEPT'
	option src 'sdwan_bond'
	option dest_port '80 443'
	option dest 'devices'
	list dest_ip '10.0.19.27'

config rule
	option target 'ACCEPT'
	option name 'Domain'
	list proto 'tcp'
	list proto 'udp'
	option dest 'domain'
	option src '*'
	option dest_port '123 53'

config rule
	option src 'guest'
	option target 'ACCEPT'
	option name 'DNS in guest'
	option dest_port '53'

config rule
	option name 'Block PTH wifi test'
	list proto 'icmp'
	option dscp 'CS7'
	option target 'REJECT'
	option src '*'
	list dest_ip 'fd00:f9a8:0:42::1'
	option direction 'in'
	option device 'wan'

config rule
	option name 'SdWan domain in'
	option dest 'domain'
	list dest_ip '10.0.53.2'
	list dest_ip 'fd00:f9a8:53:1::2'
	option target 'ACCEPT'
	option src 'sdwan_bond'
	option dest_port '547 67'

config rule
	option name 'hassio dishy'
	option src 'devices'
	list src_ip '10.0.19.27'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'
	list dest_ip '192.168.100.1'
	list dest_ip '192.168.117.1'
	option dest 'homebridge'

config rule
	list src_ip 'fd00:f9a8:fffe::2000'
	list src_ip 'fd00:f9a8:fffe::2001'
	option dest '*'
	list dest_ip 'fd00::/8'
	list dest_ip '10.0.0.0/8'
	list dest_ip '192.168.0.0/16'
	option target 'ACCEPT'
	option name 'VPN Admin'
	list proto 'all'
	option src 'sdwan_bond'

config rule
	option name 'Block domain from guest'
	list proto 'all'
	option src 'guest'
	option dest 'domain'
	option target 'DROP'

config rule
	option name 'Devices to zabbix'
	option src 'devices'
	option dest 'admin'
	option target 'ACCEPT'
	list dest_ip '10.0.18.79'
	list dest_ip 'fd00:f9a8:0:42:5054:ff:fecb:aedf'
	list proto 'all'

config rule
	option name 'Home assistant input'
	list proto 'tcp'
	option src 'devices'
	option target 'ACCEPT'
	option dest_port '443'
	list src_ip '10.0.19.27'
	list src_ip 'fd00:f9a8:0:43:be23:a1e2:a121:42cc'

config rule
	option name 'Allow-DHCP-Renew'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'
	option src 'wan'

config rule
	option name 'Allow-Ping'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'
	option src 'wan'

config rule
	option name 'Allow-IGMP'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'
	option src 'wan'

config rule
	option name 'Allow-DHCPv6'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'
	option src 'wan'

config rule
	option name 'Allow-MLD'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'
	option src 'wan'

config rule
	option name 'Allow-ICMPv6-Forward'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'
	option enabled '0'
	option src 'wan'

config rule
	option name 'Support-UDP-Traceroute'
	option dest_port '33434:33689'
	option proto 'udp'
	option family 'ipv4'
	option target 'REJECT'
	option src 'wan'

config include
	option path '/etc/firewall.user'

config include 'nat6'
	option path '/etc/firewall.nat6'
	option reload '1'

config zone
	option name 'resident'
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'ACCEPT'
	list network 'resident'

config forwarding
	option dest 'guest'
	option src 'admin'

config forwarding
	option dest 'resident'
	option src 'admin'

config rule
	list proto 'udp'
	option target 'ACCEPT'
	option src 'guest'
	option name 'Guest OpenVPN'
	option dest_port '443'

config rule
	option dest 'guest'
	option target 'DROP'
	option src_port '5353'
	option name 'block mdns to guest'

config rule
	option dest_port '5353'
	option src 'guest'
	option target 'DROP'
	option name 'block mdns guest'

config rule
	option dest_port '5353'
	option src '*'
	option target 'ACCEPT'
	option name 'allow mdns'

config forwarding
	option dest 'ovpn'
	option src 'admin'

config forwarding
	option dest 'resident'
	option src 'ovpn'

config rule
	option name 'Allow DHCP'
	option target 'ACCEPT'
	list proto 'udp'
	option src '*'
	option dest_port '67 123'

config rule
	option src 'resident'
	option name 'Openvpn'
	option target 'ACCEPT'
	list proto 'udp'
	option dest_port '443'

config rule 'glipv6_guest_dhcp'
	option name 'Allow-DHCP-IPV6'
	option src 'guest'
	option target 'ACCEPT'
	option proto 'udp'
	option dest_port '547:548'
	option family 'ipv6'

config forwarding
	option src 'admin'
	option dest 'wan'

config zone
	option name 'devices'
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'ACCEPT'
	list network 'devices'

config zone
	option name 'guest'
	option output 'ACCEPT'
	option input 'REJECT'
	option forward 'ACCEPT'
	list network 'guest'

config forwarding
	option src 'admin'
	option dest 'domain'

config forwarding
	option src 'admin'
	option dest 'devices'

config forwarding
	option src 'resident'
	option dest 'devices'

config forwarding
	option src 'ovpn'
	option dest 'devices'

config ipset
	option enabled '1'
	option name 'voice-traffic'
	option family 'ipv4'
	list match 'src_ip'
	list match 'dest_ip'

config ipset
	option name 'bypassvpn'
	option enabled '1'
	option family 'ipv4'
	option match 'dst_ip'

config rule
	list src_ip 'fd00:f9a8:0:2::/64'
	list src_ip '10.0.2.0/24'
	option dest '*'
	option target 'ACCEPT'
	option name 'Admin SDWAN'
	list proto 'all'
	option src 'sdwan_bond'

config rule
	list src_ip 'fd00:f9a8:0:10::/64'
	list src_ip '10.0.10.0/24'
	option dest 'resident'
	option target 'ACCEPT'
	option name 'Resident SDWAN'
	list proto 'all'
	option src 'sdwan_bond'

config rule
	option src 'ovpn'
	list src_ip 'fd00:f9a8:fffd::2000'
	list src_ip 'fd00:f9a8:fffd::2001'
	list src_ip '10.234.3.200'
	list src_ip '10.234.3.201'
	option dest '*'
	option target 'ACCEPT'
	option name 'Admin OVPN'
	list proto 'all'

config rule
	option src 'ovpn'
	option target 'ACCEPT'
	list src_ip 'fd00:f9a8:fffd::2000'
	list src_ip 'fd00:f9a8:fffd::2001'
	list src_ip '10.234.3.200'
	list src_ip '10.234.3.201'
	option name 'Admin OVPN Input'
	list proto 'all'

config rule
	list proto 'tcp'
	option dest_port '179'
	option target 'ACCEPT'
	option name 'BGP SDWAN'
	option src 'sdwan_bond'

config rule
	list src_ip 'fd00:f9a8:0:2::/64'
	list src_ip '10.0.2.0/24'
	option target 'ACCEPT'
	option name 'ADMIN SDWAN'
	list proto 'all'
	option src 'sdwan_bond'

config rule
	list src_ip 'fd00:f9a8:fffe::2000'
	option target 'ACCEPT'
	option name 'ADMIN SDWAN Input'
	list proto 'all'
	option src 'sdwan_bond'

config forwarding
	option src 'admin'
	option dest 'sdwan_bond'

config forwarding
	option src 'devices'
	option dest 'sdwan_bond'

config forwarding
	option src 'domain'
	option dest 'sdwan_bond'

config forwarding
	option src 'guest'
	option dest 'sdwan_bond'

config forwarding
	option src 'ovpn'
	option dest 'sdwan_bond'

config forwarding
	option src 'resident'
	option dest 'sdwan_bond'

config include 'pbr'
	option fw4_compatible '1'
	option type 'script'
	option path '/usr/share/pbr/pbr.firewall.include'

config rule
	list proto 'udp'
	option target 'REJECT'
	option dest 'sdwan_bond'
	option name 'Block tunnel nesting'
	list dest_ip '167.88.114.36'
	option dest_port '443 9468 9469'

config nat
	option src 'wan'
	option target 'MASQUERADE'
	option name 'NetgearModem'
	list proto 'all'
	option dest_ip '192.168.5.1'

config rule
	option name 'Allow ICMP'
	list proto 'icmp'
	option src '*'
	option dest '*'
	option target 'ACCEPT'

config rule
	list proto 'icmp'
	option src '*'
	option target 'ACCEPT'
	option name 'Allow ICMP'

config rule
	option name 'Hassos to Zabbix'
	list proto 'tcp'
	option src 'devices'
	option dest_port '443'
	option target 'ACCEPT'
	list src_ip 'fd00:f9a8:0:43:be23:a1e2:a121:42cc'
	list src_ip '10.0.19.27'
	list dest_ip 'fd00:f9a8:0:42:5054:ff:fecb:aedf'
	list dest_ip '10.0.18.79'

config rule
	option name 'ovpntest'
	list proto 'udp'
	option src 'resident'
	option target 'ACCEPT'
	option dest_port '443'

config rule
	option name 'Domain In'
	option src 'domain'
	option dest_port '53'
	option target 'ACCEPT'

config rule
	option name 'ATT WiFi Calling'
	list proto 'udp'
	option src 'guest'
	option dest 'sdwan_bond'
	list dest_ip '107.122.31.71'
	option dest_port '4500'
	option target 'DSCP'
	option set_dscp 'CS4'

config forwarding
	option src 'admin'
	option dest 'homebridge'

config forwarding
	option src 'devices'
	option dest 'homebridge'

config forwarding
	option src 'domain'
	option dest 'homebridge'

config forwarding
	option src 'guest'
	option dest 'homebridge'

config forwarding
	option src 'ovpn'
	option dest 'homebridge'

config forwarding
	option src 'resident'
	option dest 'homebridge'

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
7: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.255.0.66/30 brd 10.255.0.67 scope global wan
       valid_lft forever preferred_lft forever
9: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.23.32.2/32 brd 255.255.255.255 scope global dummy0
       valid_lft forever preferred_lft forever
16: br-domain: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.53.1/29 brd 10.0.53.7 scope global br-domain
       valid_lft forever preferred_lft forever
17: br-admin: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.18.1/24 brd 10.0.18.255 scope global br-admin
       valid_lft forever preferred_lft forever
21: br-devices: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.19.1/24 brd 10.0.19.255 scope global br-devices
       valid_lft forever preferred_lft forever
23: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.23.1/24 brd 10.0.23.255 scope global br-guest
       valid_lft forever preferred_lft forever
25: br-resident: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.22.1/24 brd 10.0.22.255 scope global br-resident
       valid_lft forever preferred_lft forever
28: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.33.23.2/30 brd 10.33.23.3 scope global wg0
       valid_lft forever preferred_lft forever
31: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1378 qdisc fq_codel state UNKNOWN group default qlen 500
    inet 10.234.3.1/24 scope global tun0
       valid_lft forever preferred_lft forever
10.33.23.0/30 dev wg0 table 54 proto static scope link 
default nhid 7138 via 10.255.0.65 dev wan proto 196 metric 20 
10.0.18.0/24 dev br-admin proto kernel scope link src 10.0.18.1 
10.0.19.0/24 dev br-devices proto kernel scope link src 10.0.19.1 
10.0.22.0/24 dev br-resident proto kernel scope link src 10.0.22.1 
10.0.23.0/24 dev br-guest proto kernel scope link src 10.0.23.1 
10.0.26.0/24 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.0.27.0/24 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.0.30.0/24 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.0.31.0/24 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.0.53.0/29 dev br-domain proto kernel scope link src 10.0.53.1 
10.0.53.16/29 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.23.22.1 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.23.32.1 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.33.23.4/30 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.33.23.8/30 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.33.23.12/30 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.234.3.0/24 dev tun0 proto kernel scope link src 10.234.3.1 
10.234.4.0/24 nhid 7138 via 10.255.0.65 dev wan proto ospf metric 20 
10.255.0.64/30 dev wan proto kernel scope link src 10.255.0.66 
0:	from all lookup local
1:	from all fwmark 0x34dd lookup 55
2:	from all to 192.168.5.1 lookup 55
10000:	from 10.33.23.2 lookup 54
20000:	from all to 10.33.23.2/30 lookup 54
32766:	from all lookup main
32767:	from all lookup default
90028:	from all iif lo lookup 54
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1504 state UP qlen 1000
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
7: wan@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:9a7e:aedc::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::fc43:9abd:5c33:5933/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::daec:5eff:fe8d:6940/64 scope link 
       valid_lft forever preferred_lft forever
9: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:399::3/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::7453:77ff:fe50:b69b/64 scope link 
       valid_lft forever preferred_lft forever
16: br-domain: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:53:1::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::fc4c:a7ff:fe78:2f41/64 scope link 
       valid_lft forever preferred_lft forever
17: br-admin: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:42::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
18: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
21: br-devices: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:43::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
23: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:46::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
25: br-resident: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:45::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::daec:5eff:fe8d:6941/64 scope link 
       valid_lft forever preferred_lft forever
28: wg0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:300::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::6331:aabc:dd31:2e9e/64 scope link 
       valid_lft forever preferred_lft forever
29: wlan0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::daec:5eff:fe8d:6942/64 scope link 
       valid_lft forever preferred_lft forever
30: wlan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::daec:5eff:fe8d:6943/64 scope link 
       valid_lft forever preferred_lft forever
31: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1378 state UNKNOWN qlen 500
    inet6 fd00:f9a8:fffd::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::2029:c942:3bac:1343/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever
32: wlan0-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::d8ec:5eff:fe8d:6942/64 scope link 
       valid_lft forever preferred_lft forever
33: wlan0-2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::dcec:5eff:fe8d:6942/64 scope link 
       valid_lft forever preferred_lft forever
34: vethx22Ny5@lan1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fc4c:a7ff:fe78:2f41/64 scope link 
       valid_lft forever preferred_lft forever
35: wlan1-1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::d8ec:5eff:fe8d:6943/64 scope link 
       valid_lft forever preferred_lft forever
36: wlan1-2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::dcec:5eff:fe8d:6943/64 scope link 
       valid_lft forever preferred_lft forever
fd00:f9a8:9a7e:300::/64 dev wg0 table 54 proto static metric 1024 pref medium
fe80::/64 dev wg0 table 54 proto static metric 1024 pref medium
fd00:f9a8:0:42::/64 dev br-admin proto kernel metric 256 pref medium
fd00:f9a8:0:43::/64 dev br-devices proto kernel metric 256 pref medium
fd00:f9a8:0:45::/64 dev br-resident proto kernel metric 256 pref medium
fd00:f9a8:0:46::/64 dev br-guest proto kernel metric 256 pref medium
fd00:f9a8:0:62::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:0:63::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:53:1::/64 dev br-domain proto kernel metric 256 pref medium
fd00:f9a8:53:3::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:301::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:302::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:303::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::1 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::2 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::3 dev dummy0 proto kernel metric 256 pref medium
fd00:f9a8:9a7e:aedc::/64 dev wan proto kernel metric 256 pref medium
fd00:f9a8:fffc::/64 nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto ospf metric 20 pref medium
fd00:f9a8:fffd::/64 dev tun0 proto kernel metric 256 pref medium
default nhid 7133 via fe80::ce22:abd3:3dc6:6043 dev wan proto 196 metric 20 pref medium
local ::1 dev lo table local proto kernel metric 0 pref medium
0:	from all lookup local
10000:	from fd00:f9a8:9a7e:300::2 lookup 54
10000:	from fe80::6331:aabc:dd31:2e9e lookup 54
20000:	from all to fd00:f9a8:9a7e:300::2/64 lookup 54
20000:	from all to fe80::6331:aabc:dd31:2e9e/64 lookup 54
32766:	from all lookup main
lrwxrwxrwx    1 root     root            16 Apr  9 04:27 /etc/resolv.conf -> /tmp/resolv.conf
-rw-r--r--    1 root     root            76 Apr 16 23:11 /tmp/resolv.conf
-rw-r--r--    1 root     root             0 Apr 16 19:37 /tmp/resolv.conf.d/resolv.conf.auto

/tmp/resolv.conf.d:
-rw-r--r--    1 root     root             0 Apr 16 19:37 resolv.conf.auto
==> /etc/resolv.conf <==
search domain.net
nameserver fd00:f9a8:53:1::2
nameserver fd00:f9a8:53:2::10

==> /tmp/resolv.conf <==
search domain.net
nameserver fd00:f9a8:53:1::2
nameserver fd00:f9a8:53:2::10

==> /tmp/resolv.conf.d <==
head: /tmp/resolv.conf.d: I/O error

==> /tmp/resolv.conf.d/resolv.conf.auto <==

RPI router

	"kernel": "5.10.176",
	"hostname": "nppthwangw01.domain.net",
	"system": "ARMv8 Processor rev 3",
	"model": "Raspberry Pi Compute Module 4 Rev 1.0",
	"board_name": "raspberrypi,4-compute-module",
	"rootfs_type": "ext4",
	"release": {
		"distribution": "OpenWrt",
		"version": "22.03.4",
		"revision": "r20123-38ccc47687",
		"target": "bcm27xx/bcm2711",
		"description": "OpenWrt 22.03.4 r20123-38ccc47687"
	}
}
package network

config interface 'loopback'
	option device 'lo'
	option proto 'static'
	option ipaddr '127.0.0.1'
	option netmask '255.0.0.0'

config globals 'globals'
	option packet_steering '1'

config interface 'wan'
	option device 'eth0'
	option proto 'static'
	list ipaddr '172.16.97.2/30'
	option gateway '172.16.97.1'
	option ip4table '54'
	option ip6table '54'
	option delegate '0'

config interface 'homebridge'
	option proto 'static'
	option defaultroute '0'
	option delegate '0'
	list ipaddr '10.255.0.65/30'
	list ip6addr 'fd00:f9a8:9a7e:aedc::2/64'
	list ip6addr 'fe80::ce22:abd3:3dc6:6043/64'
	option device 'eth1.1024'

config interface 'vbond0'
	option proto 'wireguard'
	option private_key ''
	option nohostroute '1'
	option peerdns '0'
	option ip4table '54'
	option ip6table '54'
	option delegate '0'
	option fwmark '0x34dd'
	list addresses '10.33.23.2/30'
	list addresses 'fd00:f9a8:9a7e:300::2/64'
	list addresses 'fe80::6331:aabc:dd31:2e9e/64'
	option mtu '1340'
	option defaultroute '0'

config wireguard_vbond0
	option description 'nvrnsdwangw01-wan'
	option public_key ''
	option preshared_key ''
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option endpoint_host ''
	option endpoint_port '9468'

config interface 'vbond1'
	option proto 'wireguard'
	option private_key ''
	option nohostroute '1'
	option defaultroute '0'
	option peerdns '0'
	option ip4table '55'
	option ip6table '55'
	option fwmark '0x34ee'
	option delegate '0'
	list addresses '10.33.23.6/30'
	list addresses 'fd00:f9a8:9a7e:301::2/64'
	list addresses 'fe80::300d:af96:223c:1ad2/24'
	option mtu '1340'

config wireguard_vbond1
	option description 'nvrnsdwangw01-wwan'
	option public_key ''
	option preshared_key ''
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option endpoint_host ''
	option endpoint_port '9469'

config interface 'vbond2'
	option proto 'wireguard'
	option private_key ''
	option nohostroute '1'
	option defaultroute '0'
	option peerdns '0'
	option ip4table '56'
	option ip6table '56'
	option fwmark '0x34ff'
	option delegate '0'
	list addresses '10.33.23.10/30'
	list addresses 'fd00:f9a8:9a7e:302::2/64'
	list addresses 'fe80::dde9:2434:26fa:2bc0/64'
	option mtu '1340'

config wireguard_vbond2
	option public_key ''
	option preshared_key ''
	option description 'nvrnsdwangw01'
	list allowed_ips '0.0.0.0/0'
	list allowed_ips '::/0'
	option endpoint_host ''
	option persistent_keepalive '25'
	option endpoint_port '443'

config device
	option type 'bridge'
	option name 'br-switch'
	list ports 'eth1'

config bridge-vlan
	option device 'br-switch'
	option vlan '2'
	list ports 'eth1:t'

config bridge-vlan
	option device 'br-switch'
	option vlan '3'
	list ports 'eth1:t'

config bridge-vlan
	option device 'br-switch'
	option vlan '10'
	list ports 'eth1:t'

config bridge-vlan
	option device 'br-switch'
	option vlan '11'
	list ports 'eth1:t'

config interface 'admin'
	option proto 'static'
	option defaultroute '0'
	list ipaddr '10.0.26.1/24'
	option device 'br-admin'
	list ip6addr 'fd00:f9a8:0:62::1/64'

config interface 'devices'
	option proto 'static'
	option delegate '0'
	option defaultroute '0'
	list ipaddr '10.0.27.1/24'
	option device 'br-devices'
	list ip6addr 'fd00:f9a8:0:63::1/64'

config interface 'resident'
	option proto 'static'
	option defaultroute '0'
	option delegate '0'
	list ipaddr '10.0.30.1/24'
	option device 'br-resident'

config interface 'guest'
	option proto 'static'
	option defaultroute '0'
	option delegate '0'
	list ipaddr '10.0.31.1/24'
	option device 'br-guest'

config device
	option type 'bridge'
	option name 'br-domain'
	option bridge_empty '1'

config interface 'domain'
	option proto 'static'
	option device 'br-domain'
	option defaultroute '0'
	option delegate '0'
	list ipaddr '10.0.53.17/29'
	list ip6addr 'fd00:f9a8:53:3::1/64'

config interface 'docker'
	option device 'docker0'
	option proto 'none'
	option auto '0'

config device
	option type 'bridge'
	option name 'docker0'

config interface 'wwan0'
	option proto 'dhcp'
	option device 'eth2'
	option peerdns '0'
	option ip4table '55'
	option ip6table '55'
	option delegate '0'

config interface 'wwan1'
	option proto 'dhcp'
	option peerdns '0'
	option ip4table '56'
	option ip6table '56'

config device
	option type 'bridge'
	option name 'br-admin'
	list ports 'br-switch.2'
	list ports 'eth0.2'

config device
	option type '8021q'
	option ifname 'eth0'
	option vid '2'
	option name 'eth0.2'

config device
	option type 'bridge'
	option name 'br-resident'
	list ports 'br-switch.10'

config device
	option type 'bridge'
	option name 'br-devices'
	list ports 'br-switch.3'

config device
	option type 'bridge'
	option name 'br-guest'
	list ports 'br-switch.11'

config rule
	option lookup '54'
	option mark '0x34dd'

config rule
	option mark '0x34ee'
	option lookup '55'

config rule
	option lookup '56'
	option mark '0x34ff'

config rule
	option dest '192.168.100.1/32'
	option lookup '54'

config rule
	option dest '192.168.1.1/32'
	option lookup '54'

config rule
	option dest '192.168.117.1/32'
	option lookup '55'

config interface 'dummy'
	option proto 'static'
	option device 'dummy0'
	list ipaddr '10.23.32.1/32'
	list ip6addr 'fd00:f9a8:9a7e:399::2/128'

config device
	option type '8021q'
	option ifname 'eth1'
	option vid '1024'
	option name 'eth1.1024'

package wireless

config wifi-device 'radio0'
	option type 'mac80211'
	option path 'platform/soc/fe300000.mmcnr/mmc_host/mmc1/mmc1:0001/mmc1:0001:1'
	option country 'US'
	option cell_density '0'
	option htmode 'VHT20'
	option band '5g'
	option channel 'auto'

package dhcp

config dnsmasq
	option domainneeded '1'
	option localise_queries '1'
	option local '/lan/'
	option domain 'lan'
	option expandhosts '1'
	option authoritative '1'
	option readethers '1'
	option ednspacket_max '1232'
	list server '1.1.1.1'
	list server '1.0.0.1'
	option rebind_protection '0'
	option leasefile '/mnt/app/dhcp.leases'
	option confdir '/mnt/app/dnsmasq'
	option localservice '0'
	list interface 'domain'
	list interface 'br-admin'
	list interface 'br-devices'
	list interface 'br-resident'
	list interface 'br-guest'
	list notinterface 'wan'
	list notinterface 'wwan0'
	list notinterface 'wwan1'
	option noresolv '1'

config dhcp 'wan'
	option interface 'wan'
	option ignore '1'
	option start '100'
	option limit '150'
	option leasetime '12h'

config odhcpd 'odhcpd'
	option maindhcp '0'
	option leasefile '/tmp/hosts/odhcpd'
	option leasetrigger '/usr/sbin/odhcpd-update'
	option loglevel '4'

config dhcp 'homebridge'
	option interface 'homebridge'
	option leasetime '12h'
	option start '64'
	option limit '64'

config dhcp 'admin'
	option interface 'admin'
	option start '100'
	option limit '150'
	option leasetime '12h'
	list dhcp_option '15,domain.net'
	list dhcp_option '6,10.0.53.18,10.0.53.10'
	option ra 'server'
	option ra_default '2'
	option dhcpv6 'server'
	list dns 'fd00:f9a8:53:3::18'
	list dns 'fd00:f9a8:53:2::10'
	list domain 'domain.net'

config dhcp 'devices'
	option interface 'devices'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'guest'
	option interface 'guest'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'resident'
	option interface 'resident'
	option start '100'
	option limit '150'
	option leasetime '12h'

config dhcp 'domain'
	option interface 'domain'
	option start '100'
	option limit '150'
	option leasetime '12h'
	option ra 'server'
	option ra_default '2'
	option dhcpv6 'server'
	list domain 'domain.net'

package firewall

config defaults
	option output 'ACCEPT'
	option synflood_protect '1'
	option input 'DROP'
	option forward 'DROP'
	option drop_invalid '1'

config zone
	option input 'DROP'
	option forward 'DROP'
	option name 'wan'
	list network 'wan'
	list network 'wwan0'
	list network 'wwan1'
	option output 'DROP'

config zone
	option name 'sdwan_bond'
	option output 'ACCEPT'
	option masq_allow_invalid '1'
	option mtu_fix '1'
	option log '1'
	list network 'dummy'
	list network 'vbond0'
	list network 'vbond1'
	list network 'vbond2'
	option input 'DROP'
	option forward 'DROP'

config zone
	option name 'ovpn'
	option output 'ACCEPT'
	option mtu_fix '1'
	option log '1'
	option input 'ACCEPT'
	list network 'ovpn'
	option forward 'REJECT'

config zone
	option name 'admin'
	option input 'ACCEPT'
	option output 'ACCEPT'
	option forward 'ACCEPT'
	list network 'admin'
	list network 'homebridge'

config zone
	option name 'domain'
	option output 'ACCEPT'
	list network 'domain'
	option forward 'REJECT'
	option input 'REJECT'
	option log '1'

config forwarding
	option src 'admin'
	option dest 'domain'

config forwarding
	option src 'resident'
	option dest 'devices'

config forwarding
	option src 'ovpn'
	option dest 'devices'

config forwarding
	option dest 'ovpn'
	option src 'admin'

config forwarding
	option src 'admin'
	option dest 'sdwan_bond'

config forwarding
	option src 'devices'
	option dest 'sdwan_bond'

config forwarding
	option src 'domain'
	option dest 'sdwan_bond'

config forwarding
	option src 'guest'
	option dest 'sdwan_bond'

config forwarding
	option src 'ovpn'
	option dest 'sdwan_bond'

config forwarding
	option src 'resident'
	option dest 'sdwan_bond'

config rule
	option name 'BFD'
	option src 'sdwan_bond'
	option target 'ACCEPT'
	option dest_port '4784 3784 3785'

config rule
	option name 'OSPF WAN'
	list proto '89'
	option target 'ACCEPT'
	option src 'sdwan_bond'

config rule
	option name 'OSPF IN allow'
	option direction 'in'
	option device 'br-switch.1024'
	list proto 'ospf'
	option src 'wan'
	option target 'ACCEPT'

config rule
	option direction 'out'
	option name 'Block sdwan wan'
	option dest 'wan'
	option mark '0x34dd'
	option target 'ACCEPT'
	option device 'eth0'
	list proto 'all'

config rule
	option direction 'out'
	option name 'Block sdwan wwan'
	option dest 'wan'
	option target 'ACCEPT'
	option mark '0x34ee'
	option device 'eth2'
	list proto 'all'

config rule
	option direction 'out'
	option device 'wwan2'
	option target 'ACCEPT'
	option mark '0x34ff'
	option dest 'wan'
	option name 'Block sdwan wwan2'
	list proto 'all'

config rule
	option name 'SDWAN HAProxy'
	list proto 'tcp'
	option target 'ACCEPT'
	option src 'sdwan_bond'
	option dest_port '80 443'
	option dest 'devices'
	list dest_ip '10.0.19.27'

config rule
	option src 'domain'
	option dest_port '53'
	option target 'ACCEPT'
	list proto 'tcp'
	list proto 'udp'

config rule
	option target 'ACCEPT'
	option name 'Domain'
	list proto 'tcp'
	list proto 'udp'
	option dest 'domain'
	option src '*'
	option dest_port '123 53'

config rule
	option src 'guest'
	option target 'ACCEPT'
	option name 'DNS in guest'
	option dest_port '53'

config rule
	option name 'ATT WiFi Calling'
	list proto 'udp'
	option src 'guest'
	option dest 'sdwan_bond'
	list dest_ip '107.122.31.71'
	option dest_port '4500'
	option target 'DSCP'
	option set_dscp 'CS4'

config nat
	option name 'StarlinkRouter'
	option dest_ip '192.168.1.1'
	option target 'MASQUERADE'
	list proto 'all'
	option src 'wan'

config nat
	option name 'ATTPhone'
	list proto 'all'
	option src 'wan'
	option dest_ip '192.168.117.1'
	option target 'MASQUERADE'

config rule
	list proto 'udp'
	option target 'REJECT'
	option dest 'sdwan_bond'
	option name 'Block tunnel nesting'
	list dest_ip '167.88.114.36'
	option dest_port '443 9468 9469'

config rule
	option name 'Allow ICMP In'
	list proto 'icmp'
	option src '*'
	option target 'ACCEPT'

config rule
	option name 'Allow ICMP'
	list proto 'icmp'
	option src '*'
	option dest '*'
	option target 'ACCEPT'

config nat
	option name 'StarlinkModem'
	list proto 'all'
	option src 'wan'
	option dest_ip '192.168.100.1'
	option target 'MASQUERADE'

config rule
	option name 'Allow-DHCP-Renew'
	option src 'wan'
	option proto 'udp'
	option dest_port '68'
	option target 'ACCEPT'
	option family 'ipv4'

config rule
	option name 'Allow-Ping'
	option src 'wan'
	option proto 'icmp'
	option icmp_type 'echo-request'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-IGMP'
	option src 'wan'
	option proto 'igmp'
	option family 'ipv4'
	option target 'ACCEPT'

config rule
	option name 'Allow-DHCPv6'
	option src 'wan'
	option proto 'udp'
	option dest_port '546'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-MLD'
	option src 'wan'
	option proto 'icmp'
	option src_ip 'fe80::/10'
	list icmp_type '130/0'
	list icmp_type '131/0'
	list icmp_type '132/0'
	list icmp_type '143/0'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Input'
	option src 'wan'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	list icmp_type 'router-solicitation'
	list icmp_type 'neighbour-solicitation'
	list icmp_type 'router-advertisement'
	list icmp_type 'neighbour-advertisement'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config rule
	option name 'Allow-ICMPv6-Forward'
	option src 'wan'
	option dest '*'
	option proto 'icmp'
	list icmp_type 'echo-request'
	list icmp_type 'echo-reply'
	list icmp_type 'destination-unreachable'
	list icmp_type 'packet-too-big'
	list icmp_type 'time-exceeded'
	list icmp_type 'bad-header'
	list icmp_type 'unknown-header-type'
	option limit '1000/sec'
	option family 'ipv6'
	option target 'ACCEPT'

config zone
	option name 'bridge'
	option input 'ACCEPT'
	option output 'ACCEPT'
	list network 'homebridge'
	option forward 'ACCEPT'
	option log '1'

config forwarding
	option src 'admin'
	option dest 'bridge'

config zone
	option name 'devices'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'devices'
	option input 'REJECT'

config zone
	option name 'resident'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'resident'
	option input 'REJECT'

config zone
	option name 'guest'
	option output 'ACCEPT'
	option forward 'REJECT'
	list network 'guest'
	option input 'REJECT'

config forwarding
	option src 'admin'
	option dest 'devices'

config forwarding
	option src 'admin'
	option dest 'guest'

config forwarding
	option src 'admin'
	option dest 'resident'

config zone 'docker'
	option output 'ACCEPT'
	option name 'docker'
	list network 'docker'
	option input 'REJECT'
	option forward 'REJECT'

config rule
	list proto 'tcp'
	option src '*'
	option dest 'wan'
	list dest_ip '192.168.117.1'
	list dest_ip '192.168.100.1'
	list dest_ip '192.168.1.1'
	option target 'ACCEPT'

config rule
	list proto 'all'
	option src '*'
	list src_ip 'fd00:f9a8:0:2::/64'
	list src_ip 'fd00:f9a8:0:42::/64'
	list src_ip 'fd00:f9a8:0:62::/64'
	list src_ip 'fd00:f9a8:fffd::/64'
	list src_ip 'fd00:f9a8:fffe::/64'
	list src_ip 'fd00:f9a8:ffff::/64'
	list src_ip 'fd00:f9a8:fffc::/64'
	option dest '*'
	option target 'ACCEPT'

config rule
	option src '*'
	list src_ip 'fd00:f9a8:0:2::/64'
	list src_ip 'fd00:f9a8:0:42::/64'
	list src_ip 'fd00:f9a8:0:62::/64'
	list src_ip 'fd00:f9a8:fffd::/64'
	list src_ip 'fd00:f9a8:fffe::/64'
	list src_ip 'fd00:f9a8:ffff::/64'
	list src_ip 'fd00:f9a8:fffc::/64'
	option target 'ACCEPT'
	list proto 'all'

config forwarding
	option src 'bridge'
	option dest 'sdwan_bond'

config forwarding
	option src 'docker'
	option dest 'sdwan_bond'

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    inet 172.16.97.2/30 brd 172.16.97.3 scope global eth0
       valid_lft forever preferred_lft forever
3: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.23.32.1/32 brd 255.255.255.255 scope global dummy0
       valid_lft forever preferred_lft forever
5: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    inet 192.168.117.47/24 brd 192.168.117.255 scope global eth2
       valid_lft forever preferred_lft forever
10: br-domain: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.53.17/29 brd 10.0.53.23 scope global br-domain
       valid_lft forever preferred_lft forever
14: br-admin: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.26.1/24 brd 10.0.26.255 scope global br-admin
       valid_lft forever preferred_lft forever
16: br-devices: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.27.1/24 brd 10.0.27.255 scope global br-devices
       valid_lft forever preferred_lft forever
18: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.31.1/24 brd 10.0.31.255 scope global br-guest
       valid_lft forever preferred_lft forever
20: br-resident: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.0.30.1/24 brd 10.0.30.255 scope global br-resident
       valid_lft forever preferred_lft forever
23: vbond1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.33.23.6/30 brd 10.33.23.7 scope global vbond1
       valid_lft forever preferred_lft forever
24: vbond2: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.33.23.10/30 brd 10.33.23.11 scope global vbond2
       valid_lft forever preferred_lft forever
25: vbond0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 qdisc noqueue state UNKNOWN group default qlen 1000
    inet 10.33.23.2/30 brd 10.33.23.3 scope global vbond0
       valid_lft forever preferred_lft forever
27: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default 
    inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0
       valid_lft forever preferred_lft forever
43: eth1.1024@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    inet 10.255.0.65/30 brd 10.255.0.67 scope global eth1.1024
       valid_lft forever preferred_lft forever
44: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1378 qdisc fq_codel state UNKNOWN group default qlen 500
    inet 10.234.4.1/24 scope global tun0
       valid_lft forever preferred_lft forever
default nhid 4327 dev eth0 table 10000 proto 195 metric 20 
default via 172.16.97.1 dev eth0 table 54 proto static 
10.33.23.0/30 dev vbond0 table 54 proto static scope link 
172.16.97.0/30 dev eth0 table 54 proto static scope link 
default via 192.168.117.1 dev eth2 table 55 proto static src 192.168.117.47 
10.33.23.4/30 dev vbond1 table 55 proto static scope link 
192.168.117.0/24 dev eth2 table 55 proto static scope link 
10.33.23.8/30 dev vbond2 table 56 proto static scope link 
default nhid 11806 via 10.33.23.1 dev vbond0 proto 196 metric 20 
10.0.18.0/24 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.0.19.0/24 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.0.22.0/24 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.0.23.0/24 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.0.26.0/24 dev br-admin proto kernel scope link src 10.0.26.1 
10.0.27.0/24 dev br-devices proto kernel scope link src 10.0.27.1 
10.0.30.0/24 dev br-resident proto kernel scope link src 10.0.30.1 
10.0.31.0/24 dev br-guest proto kernel scope link src 10.0.31.1 
10.0.53.0/29 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.0.53.16/29 dev br-domain proto kernel scope link src 10.0.53.17 
10.23.22.1 nhid 11806 via 10.33.23.1 dev vbond0 proto ospf metric 20 
10.23.32.2 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.33.23.12/30 nhid 11806 via 10.33.23.1 dev vbond0 proto ospf metric 20 
10.234.3.0/24 nhid 9997 via 10.255.0.66 dev eth1.1024 proto ospf metric 20 
10.234.4.0/24 dev tun0 proto kernel scope link src 10.234.4.1 
10.255.0.64/30 dev eth1.1024 proto kernel scope link src 10.255.0.65 
172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown  
0:	from all lookup local
1:	from all fwmark 0x34dd lookup 54
2:	from all fwmark 0x34ee lookup 55
3:	from all fwmark 0x34ff lookup 56
4:	from all to 192.168.100.1 lookup 54
5:	from all to 192.168.1.1 lookup 54
6:	from all to 192.168.117.1 lookup 55
10000:	from 172.16.97.2 lookup 54
10000:	from 10.33.23.6 lookup 55
10000:	from 10.33.23.2 lookup 54
10000:	from 10.33.23.10 lookup 56
10000:	from 192.168.117.47 lookup 55
20000:	from all to 172.16.97.2/30 lookup 54
20000:	from all to 10.33.23.6/30 lookup 55
20000:	from all to 10.33.23.2/30 lookup 54
20000:	from all to 10.33.23.10/30 lookup 56
20000:	from all to 192.168.117.47/24 lookup 55
32766:	from all lookup main
32767:	from all lookup default
90002:	from all iif lo lookup 54
90005:	from all iif lo lookup 55
90023:	from all iif lo lookup 55
90024:	from all iif lo lookup 56
90025:	from all iif lo lookup 54
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::e65f:1ff:fe2b:5e60/64 scope link 
       valid_lft forever preferred_lft forever
3: dummy0: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:399::2/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::3862:aeff:feac:d3e8/64 scope link 
       valid_lft forever preferred_lft forever
5: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::224:9bff:fe28:d690/64 scope link 
       valid_lft forever preferred_lft forever
10: br-domain: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:53:3::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::fcb7:dfff:fe56:6de4/64 scope link 
       valid_lft forever preferred_lft forever
11: br-switch: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
14: br-admin: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:62::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
16: br-devices: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:0:63::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
18: br-guest: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
20: br-resident: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
23: vbond1: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:301::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::300d:af96:223c:1ad2/24 scope link 
       valid_lft forever preferred_lft forever
24: vbond2: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:302::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::dde9:2434:26fa:2bc0/64 scope link 
       valid_lft forever preferred_lft forever
25: vbond0: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1340 state UNKNOWN qlen 1000
    inet6 fd00:f9a8:9a7e:300::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::6331:aabc:dd31:2e9e/64 scope link 
       valid_lft forever preferred_lft forever
30: veth1sB1ZP@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fcab:41ff:fe5a:46e6/64 scope link 
       valid_lft forever preferred_lft forever
31: veth0cEhEK@eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::fc1c:ffff:fed0:851e/64 scope link 
       valid_lft forever preferred_lft forever
43: eth1.1024@eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fd00:f9a8:9a7e:aedc::2/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::ce22:abd3:3dc6:6043/64 scope link 
       valid_lft forever preferred_lft forever
    inet6 fe80::2ef7:f1ff:fe1c:97d9/64 scope link 
       valid_lft forever preferred_lft forever
44: tun0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1378 state UNKNOWN qlen 500
    inet6 fd00:f9a8:fffc::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::c2d1:2189:c09b:6647/64 scope link stable-privacy 
       valid_lft forever preferred_lft forever
default nhid 4328 dev eth0 table 10000 proto 195 metric 20 pref medium
fd00:f9a8:9a7e:300::/64 dev vbond0 table 54 proto static metric 1024 pref medium
fe80::/64 dev vbond0 table 54 proto static metric 1024 pref medium
fd00:f9a8:9a7e:301::/64 dev vbond1 table 55 proto static metric 1024 pref medium
fe80::/24 dev vbond1 table 55 proto static metric 1024 pref medium
fd00:f9a8:9a7e:302::/64 dev vbond2 table 56 proto static metric 1024 pref medium
fe80::/64 dev vbond2 table 56 proto static metric 1024 pref medium
fd00:f9a8:0:42::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:43::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:45::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:46::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:62::/64 dev br-admin proto kernel metric 256 pref medium
fd00:f9a8:0:63::/64 dev br-devices proto kernel metric 256 pref medium
fd00:f9a8:53:1::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:53:3::/64 dev br-domain proto kernel metric 256 pref medium
fd00:f9a8:9a7e:303::/64 nhid 11802 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::1 nhid 11802 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::2 dev dummy0 proto kernel metric 256 pref medium
fd00:f9a8:9a7e:399::3 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:aedc::/64 dev eth1.1024 proto kernel metric 256 pref medium
fd00:f9a8:fffc::/64 dev tun0 proto kernel metric 256 pref medium
fd00:f9a8:fffd::/64 nhid 9991 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
0:	from all lookup local
10000:	from fd00:f9a8:9a7e:301::2 lookup 55
10000:	from fe80::300d:af96:223c:1ad2 lookup 55
10000:	from fd00:f9a8:9a7e:300::2 lookup 54
10000:	from fe80::6331:aabc:dd31:2e9e lookup 54
10000:	from fd00:f9a8:9a7e:302::2 lookup 56
10000:	from fe80::dde9:2434:26fa:2bc0 lookup 56
20000:	from all to fd00:f9a8:9a7e:301::2/64 lookup 55
20000:	from all to fe80::300d:af96:223c:1ad2/24 lookup 55
20000:	from all to fd00:f9a8:9a7e:300::2/64 lookup 54
20000:	from all to fe80::6331:aabc:dd31:2e9e/64 lookup 54
20000:	from all to fd00:f9a8:9a7e:302::2/64 lookup 56
20000:	from all to fe80::dde9:2434:26fa:2bc0/64 lookup 56
32766:	from all lookup main
lrwxrwxrwx    1 root     root            16 Apr  9 04:27 /etc/resolv.conf -> /tmp/resolv.conf
-rw-r--r--    1 root     root            77 Apr 16 20:31 /tmp/resolv.conf
-rw-r--r--    1 root     root            18 Apr 16 18:50 /tmp/resolv.conf.d/resolv.conf.auto

/tmp/resolv.conf.d:
-rw-r--r--    1 root     root            18 Apr 16 18:50 resolv.conf.auto
==> /etc/resolv.conf <==
search domain.net
nameserver fd00:f9a8:53:3::18
nameserver fd00:f9a8:53:2::10

==> /tmp/resolv.conf <==
search domain.net
nameserver fd00:f9a8:53:3::18
nameserver fd00:f9a8:53:2::10

==> /tmp/resolv.conf.d <==
head: /tmp/resolv.conf.d: I/O error

==> /tmp/resolv.conf.d/resolv.conf.auto <==
# Interface wwan0

If I understand properly, traffic initiates from client vlan fd00:f9a8:0:42::/64, goes through the home router, reaches the RPi and it's dropped, right?

The RPi has a default route in table 10000, but I don't see any rule to classify the lan into that routing table, so it doesn't have any uplink.
There is also a mistake fe80::/24 dev vbond1 table 55 proto static metric 1024 pref medium should be /64.
Furthermore since you are using ULAs I don't see the point of hardcoding the LLs.

Wireguard uses the public key to identify a peer, so having duplicate keys within your network does not work. Generate a new key pair for each peer.

Also of course the IP addresses must be unique. I'm not sure what you're trying to do with ULAs. I do know that Wireguard will work with just link locals on each peer, but it does not automatically assign them you must place them in the config. If you were to use ULAs between sites, they should be in the same /64 prefix on both sides.

If you are using ULAs at some point they will have to be NATd to a GUA to reach the Internet. I see a ULA on one of your wan interfaces, which is pointless since an ISP will not route it.

1 Like

That is the issue, traffic from rpi to ipv6 default route works. Traffic from home router out its cell modem tunnel works, but when its directed across the rpi, its seen coming in eth1.1024 and then i cant find it being sent unidirectional on any interface

I used ULAs because in the past i found it useful to test, only recently did i discover that adding LL allows ipv6 ospf. I corrected the /24, unfortunatley that didnt change anything.
Also, looking at the ::/0 i see the default route i setup in thr main routing table for vbond0, which is why the rpi router can ping out. I dont think the other ::/0 matters.

Ive enabled debugging on all of the zones and dont see any firewall logs related to my traffic. Im not sure if it would be possible to log all dropped netfilter connections to rule out firewall completely.

I nat 66 my whole vpn mesh out of the vps, so this is not an issue. But the other problem is the sdwan vps also is a default gateway for the rest of the mesh network, so this traffic issue is also preventing ULA to ULA connectivity

The wireguard was physically moved to the rpi, there are no duplicate keys.

Well, thank you for that! :smiley:
I finally made my wg tunnel work with ipv6!

On your issue now. eth1.1024 is assigned to interface 'homebridge', which belongs to firewall zones admin and bridge. Fix that, or it might never be evaluated properly in the firewall rules. For example there is no forwarding from bridge to sdwan_bond.
Then, you have only this default route:
default nhid 4328 dev eth0 table 10000 proto 195 metric 20 pref medium
but I don't see any rule that the ingress fd00:... or localhost is using that table.

Here's what I see in my main routing table. There's a default route which is currently using vbond0

Thanks for the 2nd pair of eyes, I'm glad I could help you. I had been using BGP recently to work around the LL issue, but I couldn't get BFD working until I added the LL and then though, hey why not try ospf now.

Fixing the overlapping zones on bridge didn't help, I am leaving the bridge network in admin for troubleshooting, since it's supposed to allow all traffic to any other zone

ip -6 route
fd00:f9a8:0:42::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:43::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:45::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:46::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:62::/64 dev br-admin proto kernel metric 256 pref medium
fd00:f9a8:0:63::/64 dev br-devices proto kernel metric 256 pref medium
fd00:f9a8:53:1::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:53:3::/64 dev br-domain proto kernel metric 256 pref medium
fd00:f9a8:9a7e:303::/64 nhid 5120 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::1 nhid 5120 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::2 dev dummy0 proto kernel metric 256 pref medium
fd00:f9a8:9a7e:399::3 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:aedc::/64 dev eth1.1024 proto kernel metric 256 pref medium
fd00:f9a8:fffd::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-admin proto kernel metric 256 pref medium
fe80::/64 dev eth1.1024 proto kernel metric 256 pref medium
fe80::/64 dev dummy0 proto kernel metric 256 pref medium
fe80::/64 dev dummy1 proto kernel metric 256 pref medium
fe80::/64 dev br-switch proto kernel metric 256 pref medium
fe80::/64 dev br-devices proto kernel metric 256 pref medium
fe80::/64 dev br-resident proto kernel metric 256 pref medium
fe80::/64 dev br-guest proto kernel metric 256 pref medium
fe80::/64 dev vethmeLG8u proto kernel metric 256 pref medium
fe80::/64 dev vethEF9jP1 proto kernel metric 256 pref medium
fe80::/64 dev br-domain proto kernel metric 256 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium
fe80::/64 dev tun0 proto kernel metric 256 pref medium
default nhid 5120 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto 196 metric 20 pref medium

That looks better, but it's troubling me that it wasn't visible in the ip -6 ro list table all
What does ip -6 route get 2001::1 from fd00:f9a8:0:42::11 dev eth1.1024 give?

Here is the result from the RPI truck router

2001::1 from fd00:f9a8:0:42::11 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto 196 src fd00:f9a8:9a7e:300::2 metric 20 pref medium

And here's from the home router

2001::1 from fd00:f9a8:0:42::11 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto 196 src fd00:f9a8:9a7e:aedc::1 metric 20 pref medium

Both of these are expected, and I also confirmed that my routes are installed in the SDWAN vps for both the rpi and home router subnets.

You might be onto something with the concern about the lack of default route.

Here's the RPI

ip -6 ro list table all|grep eth0
default nhid 78 dev eth0 table 10000 proto 195 metric 20 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
anycast fe80:: dev eth0 table local proto kernel metric 0 pref medium
local fe80::e65f:1ff:fe2b:5e60 dev eth0 table local proto kernel metric 0 pref medium
multicast ff00::/8 dev eth0 table local proto kernel metric 256 pref medium

Here's the home router

ip -6 ro list table all|grep wan
fd00:f9a8:0:62::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:0:63::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:53:3::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:301::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:302::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:303::/64 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::1 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::2 nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto ospf metric 20 pref medium
fd00:f9a8:9a7e:aedc::/64 dev wan proto kernel metric 256 pref medium
fe80::/64 dev wan proto kernel metric 256 pref medium
default nhid 14021 via fe80::2ef7:f1ff:fe1c:97d9 dev wan proto 196 metric 20 pref medium
anycast fd00:f9a8:9a7e:aedc:: dev wan table local proto kernel metric 0 pref medium
local fd00:f9a8:9a7e:aedc::1 dev wan table local proto kernel metric 0 pref medium
anycast fe80:: dev wan table local proto kernel metric 0 pref medium
local fe80::daec:5eff:fe8d:6940 dev wan table local proto kernel metric 0 pref medium
local fe80::fc43:9abd:5c33:5933 dev wan table local proto kernel metric 0 pref medium
multicast ff00::/8 dev wan table local proto kernel metric 256 pref medium

I notice that the home router lists the wan interface as the default route and the rpi is incorrect in table 10000, I'm going to check the interface config, essentially I have each ISP uplink setup in it's own routing table, eth0 is in table 54, and eth2 is in table 55. I then use routing policy to route the tunnel traffic for each wireguard connection out the specified uplink...but no where did I specify 10000
eth0 should never have gotten an ipv6 default route, I don't enable ipv6 DHCP/SLAAC on the isp links

So I found the eth0 default route in 10000 was an errant PBR I had created in FRR without assigning to an interface. I removed that and the results look correct, however still no change in the actual ipv6 routing

fd00:f9a8:9a7e:300::/64 dev vbond0 table 54 proto static metric 1024 pref medium
fe80::/64 dev vbond0 table 54 proto static metric 1024 pref medium
fd00:f9a8:9a7e:301::/64 dev vbond1 table 55 proto static metric 1024 pref medium
fe80::/64 dev vbond1 table 55 proto static metric 1024 pref medium
fd00:f9a8:9a7e:302::/64 dev vbond2 table 56 proto static metric 1024 pref medium
fe80::/64 dev vbond2 table 56 proto static metric 1024 pref medium
fd00:f9a8:0:42::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:43::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:45::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:46::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:0:62::/64 dev br-admin proto kernel metric 256 pref medium
fd00:f9a8:0:63::/64 dev br-devices proto kernel metric 256 pref medium
fd00:f9a8:53:1::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:53:3::/64 dev br-domain proto kernel metric 256 pref medium
fd00:f9a8:9a7e:303::/64 nhid 6964 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::1 nhid 6964 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:399::2 dev dummy0 proto kernel metric 256 pref medium
fd00:f9a8:9a7e:399::3 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fd00:f9a8:9a7e:aedc::/64 dev eth1.1024 proto kernel metric 256 pref medium
fd00:f9a8:fffd::/64 nhid 375 via fe80::fc43:9abd:5c33:5933 dev eth1.1024 proto ospf metric 20 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev br-admin proto kernel metric 256 pref medium
fe80::/64 dev eth1.1024 proto kernel metric 256 pref medium
fe80::/64 dev dummy0 proto kernel metric 256 pref medium
fe80::/64 dev dummy1 proto kernel metric 256 pref medium
fe80::/64 dev br-switch proto kernel metric 256 pref medium
fe80::/64 dev br-devices proto kernel metric 256 pref medium
fe80::/64 dev br-resident proto kernel metric 256 pref medium
fe80::/64 dev br-guest proto kernel metric 256 pref medium
fe80::/64 dev vethmeLG8u proto kernel metric 256 pref medium
fe80::/64 dev vethEF9jP1 proto kernel metric 256 pref medium
fe80::/64 dev br-domain proto kernel metric 256 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium
fe80::/64 dev tun0 proto kernel metric 256 pref medium
default nhid 6964 via fe80::fda2:4dff:fe1d:3721 dev vbond0 proto 196 metric 20 pref medium

I simplified everything and put the bridge into the SDWAN zone for on the home router and rpi router sides and enabled forwarding in that zone. Still no ipv6 routing across the rpi router if it originates from the home router.
ipv4 never has had an issue, so the firewall zones dont seem to be related