WireGuard, No Internet Access

Couldn't get to which router? I think we need to put internet access to one side for a moment and get a good working site to site config in place. That gives a starting point and somewhere to fall back to if/when things don't work when trying to get the internet working.

Can you get your configs on each end setup so the peer has Allowed IPs for opposing WG interface and subnet. That should give devices at each site access to resources at the other site. If that's working then we can build on that to get internet access from the OpenWrt end via the pfSense end.

1 Like

Sorry! Should have been more clear - meant OpenWrt. Will be more careful to identify.

You are right, 100% agree!

That makes sense! I think I have that, but let me explain - I may be wrong! On the pfSense side, I have (the OpenWrt WG IP, and DHCP subnet),

192.168.253.3/32, 192.168.0.0/24

And on OpenWrt, setting is (WG subnet, internet test address, .1.x and .2.x subnets on pfSense side, and OpenWrt subnet ... seems to be required for ping FROM OpenWrt to work),

  • 192.168.253.0/24
  • 216.115.184.69/32
  • 192.168.1.0/24
  • 192.168.2.0/24
  • 192.168.0.0/24

But for example, I can't get from boxes on the .2.x subnet to machines on OpenWrt (on the 192.168.0.x subnet). Does that make sense?

Thanks!

Change 192.168.253.0/24 to the actual IP address for the WG interface on pfSense (192.168.253.1/32?)
Get rid of 216.115.184.69/32, and 192.168.0.0/24. And also remove 192.168.1.0/24 unless you have any need to access that subnet from the OpenWrt end.
Restart the WG interface after making changes.

1 Like

Done! Routing table on pfSense is much simpler now :slight_smile:

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         host254-183.res 0.0.0.0         UG    0      0        0 eth0.2
x.x.x.x         host254-183.res 255.255.255.255 UGH   0      0        0 eth0.2
128.189.183.0   *               255.255.255.0   U     0      0        0 eth0.2
192.168.2.0     *               255.255.255.0   U     0      0        0 wg0
192.168.253.1   *               255.255.255.255 UH    0      0        0 wg0

What works, what doesn't

  1. ping, OpenWrt => pfSense ... works.
  2. ping, pfSense => OpenWrt subnet device (.0.x), fails
  3. ping, pfSense => OpenWrt WG (.253.3), works

Seems as expected, except for pfSense to OpenWrt subnet ... agreed?

Thanks!

Is that router to router? Or to a device in the pfSense subnet?

So, also checked tcpdump, on the OpenWrt side (tcpdump -i wg0 icmp), Got the following,

  1. pfSense, ping 192.168.253.3. OpenWrt capture,
17:48:55.264183 IP 192.168.253.1 > 192.168.253.3: ICMP echo request, id 54567, seq 0, length 64
17:48:55.264339 IP 192.168.253.3 > 192.168.253.1: ICMP echo reply, id 54567, seq 0, length 64
17:48:56.266458 IP 192.168.253.1 > 192.168.253.3: ICMP echo request, id 54567, seq 1, length 64
17:48:56.266579 IP 192.168.253.3 > 192.168.253.1: ICMP echo reply, id 54567, seq 1, length 64
  1. pfSense, ping 192.168.0.120
17:49:11.320109 IP 192.168.253.1 > Google-Home-Mini.lan: ICMP echo request, id 7551, seq 0, length 64
17:49:12.320072 IP 192.168.253.1 > Google-Home-Mini.lan: ICMP echo request, id 7551, seq 1, length 64
17:49:13.326190 IP 192.168.253.1 > Google-Home-Mini.lan: ICMP echo request, id 7551, seq 2, length 64
17:49:14.333769 IP 192.168.253.1 > Google-Home-Mini.lan: ICMP echo request, id 7551, seq 3, length 64
17:49:15.336311 IP 192.168.253.1 > Google-Home-Mini.lan: ICMP echo request, id 7551, seq 4, length 64

Do I perhaps need to bridge wg0 to lan? Seems the traffic does get to OpenWrt, but not to the device?

Thanks!

Either one works ... from OpenWrt, I can ping pfsense WG or subnet addresses (and get a reply).

Never mind, tried that - no difference (unless a reboot is required for this to take effect?).

Thanks!

Can you post the output of uci export network; uci export firewall; ip ro from the OpenWrt router. Remove any sensitive info like public IPs or encryption keys.

You bet - attached. Thanks!

~# uci export network; uci export firewall; ip ro
package network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd7b:819c:917f::/48'

config interface 'lan'
        option proto 'static'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option ipaddr '192.168.0.1'
        option delegate '0'
        option type 'bridge'
        option ifname 'eth0.1 wg0'

config interface 'wan'
        option ifname 'eth0.2'
        option proto 'dhcp'

config device 'wan_dev'
        option name 'eth0.2'
        option macaddr 'b0:4e:26:3f:9f:31'

config switch
        option name 'switch0'
        option reset '1'
        option enable_vlan '1'

config switch_vlan
        option device 'switch0'
        option vlan '1'
        option ports '2 3 4 5 0t'

config switch_vlan
        option device 'switch0'
        option vlan '2'
        option ports '1 0t'

config interface 'wg0'
        option proto 'wireguard'
        option private_key '...'
        list addresses '192.168.253.3'
        option auto '0'

config wireguard_wg0
        option description '...'
        option public_key '...'
        option preshared_key '...'
        option route_allowed_ips '1'
        option endpoint_host '...'
        option endpoint_port '...'
        list allowed_ips '192.168.253.1/32'
        list allowed_ips '192.168.2.0/24'

package firewall

config defaults
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option synflood_protect '1'

config zone
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        option network 'lan wg0'

config zone
        option name 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'
        option device 'tun0'
        option network 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option src_ip 'fc00::/6'
        option dest_ip 'fc00::/6'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config include
        option path '/etc/firewall.user'

config redirect
        option name 'ssh'
        option src 'wan'
        option src_dport '...'
        option dest_ip '192.168.0.1'
        option target 'DNAT'
        option dest 'lan'
        list proto 'tcp'
        list proto 'udp'
        option dest_port '...'

config redirect
        option target 'DNAT'
        option src 'wan'
        option dest 'lan'
        option proto 'tcp'
        option src_dport '80'
        option dest_ip '192.168.0.1'
        option dest_port '80'
        option name 'http'
        option src_ip '192.168.0.0/16'

config redirect
        option target 'DNAT'
        option src 'wan'
        option dest 'lan'
        option proto 'tcp udp'
        option src_dport '161'
        option dest_ip '192.168.0.1'
        option dest_port '161'
        option name 'snmp'
        option src_ip '192.168.0.0/16'
        option enabled '0'

config forwarding
        option src 'lan'
        option dest 'wan'

default via ... dev eth0.2  src 128.189.183.189
... via 128.189.183.254 dev eth0.2
.../24 dev eth0.2 scope link  src 128.189.183.189
192.168.2.0/24 dev wg0 scope link
192.168.253.1 dev wg0 scope link

FYI, a bit more digging / debugging, to me it's looking like this is on the OpenWrt side (but I may be wrong!). Let me explain why I say this - by all means correct me!

And to clarify up front - I added 216.115.184.69/32 to the Allowed IPs list on OpenWrt, just so I can get to a single (internet) IP, for testing. What I tried,

  1. On OpenWrt, ping -I wg0 ip4.me, and on pfSense, tcpdump -i wg0 host ip4.me => try to ping the internet from OpenWrt, watch for traffic over the WG tunnel. The output,
17:45:04.158322 IP 192.168.253.3 > 216.115.184.69: ICMP echo request, id 21456, seq 0, length 64
17:45:04.196934 IP 216.115.184.69 > 192.168.253.3: ICMP echo reply, id 21456, seq 0, length 64
17:45:05.160106 IP 192.168.253.3 > 216.115.184.69: ICMP echo request, id 21456, seq 1, length 64
17:45:05.199040 IP 216.115.184.69 > 192.168.253.3: ICMP echo reply, id 21456, seq 1, length 64

Great! And I do get a ping reply. This is from the 192.168.253.3 address (OpenWrt WG IP).

  1. Same thing (tcpdump), but from the OpenWrt subnet (.0.x), so ping -I br-lan ip4.me => no traffic at all arriving at pfSense.

Make sense? Do I need to somehow forward the .0.x sourced traffic to the WG tunnel (on OpenWrt)? Or did I just do something stupid here (entirely possible :rofl:)?

Thanks!

Does site to site access work? I.e. can devices in the OpenWrt subnet access devices in the pfSense subnet and vice versa?

1 Like

Yep! Just can't get from the OpenWrt .0.x subnet to the internet (even to the IP specified in Allowed IPs on the OpenWrt side). It seems that traffic initiated from .0.x (OpenWrt, to pfSense) is not getting in to the WG tunnel.

Thanks!

If that were the case then you wouldn't be able to access devices in the pfSense subnet from devices in the OpenWrt subnet. So it could be that internet traffic at the OpenWrt end isn't being routed properly through the WG tunnel, but internet access wasn't working when you had 0.0.0.0/0 as the Allowed IP and the WG tunnel set as the default route. Or there's a misconfiguration at the pfSense end.

Everything seems to be working as it should be from an OpenWrt perspective. Unless you can replace the pfSense end with an OpenWrt device (if only temporarily) I'm not sure there's much further that can be done to assist you.

1 Like

Sorry! I mis-spoke earlier - I can't get from the OpenWrt subnet to pfSense. I checked ping, but I need to be careful to use -I br-lan ... or it comes from WG directly. Apologies!

No, nothing gets to pfSense, from OpenWrt .0.x. It does get there from .253.3 (WG direct).

And I don't need anything like this, right?

Trying to figure out how to get traffic to pfSense, from .0.x.

Thanks!!!

EDIT: So your comment above is correct ... "OpenWrt end isn't being routed properly through the WG tunnel".

Have you tried accessing the internet or the pfSense subnet from anything other than the router at the OpenWrt end?

1 Like

Yep! I actually have 3 other WG clients - 2 iPhones, and a laptop. All work, going through pfSense, and can get to the pfSense subnet, also the internet (redirecting all traffic, using 0.0.0.0/0). I also checked (on all 3) my public IP, and it's correct (shows my pfSense public IP). So I know it's going through the box.

And even from OpenWrt, if I use the wg0 interface (e.g. ping -I wg0 ip4.me) .. I see the traffic in tcpdump on pfSense, and I do get ping replies (from the internet). It only fails when I use the OpenWrt .0.x subnet (i.e. ping -I br-lan ip4.me), and in that case no traffic seen at pfSense (tcpdump left running).

So it really seems like traffic from .0.x is not getting on to the WG tunnel (on the OpenWrt end), agreed?

Thanks!

Are any of those other devices behind the OpenWrt router? You really need to be using a client device within the subnet to be doing this testing and not trying to fudge it with pings from router interfaces.

And, if I'm reading your previous thread correctly, when you did test with devices in the OpenWrt subnet they could access devices within the pfSense subnet. Is that correct? I'm not concerned with internet access at the moment, just access between local devices (not the routers) at each end.

1 Like

Agree with you! This may be fooling me, no argument at all. Let me try - unfortunately, the router is ~ 1800 miles away ... LOL!

Thanks!!

OK, to get back on this, keep others informed as well - seems the key issue is that NAT needs to be done ... either in OpenWrt (all traffic to pfSense using the WG IP address), or on pfSense, with multiple IP addresses (WG IP, OpenWrt DHCP subnet) across the link. As the OpenWrt box is remote (for me :laughing:), pfSense is local ... I'll mess with pfSense. Easier to undo breakage locally ... LOL!

But just to understand, it seems like I could potentially do this using the Firewall > NAT Rules on OpenWrt, right?

Thanks for all the help - so much appreciated!