How to configure wireguard clients to connect to internal services

I'm running OpenWrt 21.02-SNAPSHOT r15812+1084-46b6ee7ffc / LuCI openwrt-21.02 branch git-22.335.71649-0ecaf74 on a Gl.I-net Flint 2 router - have run on a number of different devices over the years, so I'm not new to openWRT (used to build my own for a Linksys ACM3200 for a long time).

I set up Wireguard on the router some time ago, and use it successfully with my phone, Android tablet, and another router as a client (great for being on the road). I use it in a non-split tunnel configuration - all traffic goes from the remote device through my router (I have a snappy symmetrical fiber connection).

The problem that I'm having is that I have web services that I host inside my network, and when I'm connected to the VPN, I cannot access those services - everything ends up being redirected to the router's management interface.

It seems that I need something like NAT loopback, but to go from the WG interface to the public interface. The problem, of course, is that NAT loopback is configured on a port forward, and if I set up a port forward from WG port 443 to the internal service, then all requests to any host on port 443 (e.g., https://www.google.com) end up going to my internal server, and not out to the actual internet.

It seems there ought to be a way to accomplish what I'm trying to do here, but I can't figure out what the right configuration is that keeps all traffic from my remote devices going through the VPN (I use it for adblocking while on the road) while still letting me access my internal servers' web interfaces.

Thanks for any tips/pointers.

It appears you are using firmware that is not from the official OpenWrt project.

When using forks/offshoots/vendor-specific builds that are "based on OpenWrt", there may be many differences compared to the official versions (hosted by OpenWrt.org). Some of these customizations may fundamentally change the way that OpenWrt works. You might need help from people with specific/specialized knowledge about the firmware you are using, so it is possible that advice you get here may not be useful.

You may find that the best options are:

  1. Install an official version of OpenWrt, if your device is supported (see https://firmware-selector.openwrt.org).
  2. Ask for help from the maintainer(s) or user community of the specific firmware that you are using.
  3. Provide the source code for the firmware so that users on this forum can understand how your firmware works (OpenWrt forum users are volunteers, so somebody might look at the code if they have time and are interested in your issue).

If you believe that this specific issue is common to generic/official OpenWrt and/or the maintainers of your build have indicated as such, please feel free to clarify.

You are correct that my firmware is built by Gl.Inet - but as indicated, it is openWRT firmware. I have also asked them for some assistance.

But if I were configuring this on stock openwrt, what would I do? If it's a deficiency in their build options, I would like to report it as a bug to them. I'm capable of using Luci or config file modification directly (I'm no stranger to either). I just need to know what configuration needs to be put in place - if it works, great. If it doesn't, then I'll raise an issue with the vendor so I can get it addressed.

It’s not entirely clear from where you are accessing the internal resources, but if you’re running wireguard in a server type config, it should not cause this issue. At least with standard OpenWrt. All bets are off, though, as gl-inet firmware may behave significantly differently than expected with the official openwrt firmware.

Also, to be clear, you can install Official OpenWrt on your device - it is well supported here.

Here's a diagram of what is working and what isn't:

The web server is at https://www.mydomain.net (obviously not the actual zone), and the router is configured to forward port 443 on the external to an internal port that is my reverse proxy that handles forwarding to the different services).

The forward from WAN/443 to LAN/host:port has NAT relfection turned on, so from LAN -> https://www.mydomain.net works just fine.

The Wireguard server is running on the router itself. It's configured for a different subnet than the LAN, and has its own interface (it's not part of the br-lan bridge device).

I understand that I can run the openWRT firmware here. I may do that in the future, but for the time being, I want to stick with their compiled firmware, as it says it's a stock OpenWRT firmware with a valid hash from the OpenWRT github repo (I checked that out). They may have added things on top of it, sure (like their own management UI, which has created a CORS issue with luci that is clearly their issue and not an OpenWRT issue - so I'm not asking about that here).

This is really just an iptables issue, it seems - and that works consistently regardless of whether I use OpenWRT's official build or GL.Inet's build - iptables is iptables.

OpenWrt 21.02-SNAPSHOT r15812+1084-46b6ee7ffc is https://github.com/openwrt/openwrt/commit/46b6ee7ffc

I appreciate that this isn't official firmware. I just want to know what I'm missing in the iptables/firewall configuration so I can get this working properly - and that is "stock" OpenWRT from my experience working with the official builds (and my own builds from the openwrt git repo) on other platforms for over a decade - as far as I can tell, they haven't changed anything about how that works.

Not to sound overly harsh, but if that's what you want to do then you need to go and get support from them. It's really that simple.

OpenWRT doesn't use iptables, it's moved to nftables. Along with the change from fw3 to fw4. So things are different.

2 Likes

In that case, you need to ask on their forums. This forum doesn’t provide support for gl-inet forks.

sigh

It is what it is, I guess. I was expecting better from this community. Y'all have been great in the past when I've had issues in my own self-builds, going back to the mailing lists and venues for help available prior to the move to Discourse.

21.02 (as indicated) is what's installed, and AFAICS, iptables was still the default in that version. openWRT moved in 22.03.

Even though the router says that it is using a build that is specifically tied to a valid has in the openWRT source tree? If it was a fork, that hash wouldn't be valid.

Whatever. I'm really disappointed. This should be a simple NAT/DNAT question, but apparently y'all don't want to tell me how it should work, just to chastise me for not using an official firmware build.

I get it. I support a Linux distribution, and we get all sorts of weird questions about downstream distros. It's a pain. But the way we approach it is to determine if the problem is with our stuff or if it's with the downstream provider, rather than to tell the user that they can't ask us for help.

To be clear about this, 21.02 has been eol and unsupported for several years already. In order to use it with modern devices, gl-inet makes major changes such that their fork only superficially resembles the official openwrt project. It is the equivalent of artificial “maple flavored syrup” compared to real 100% natural maple syrup.

We cannot support things that have don’t come from here because those modifications make it effectively a black box. We don’t know what they have changed, why, and how is is supposed to work with those changes.

Your device can run official openwrt, and we can support you in that context. But if you choose to run their firmware, you need to ask them for help.

1 Like

Fine, whatever. I still think that a pointer on how it should work with iptables would be sufficient to get me in the right direction, but since I can't even get that much of a pointer here, I'll no longer bother you.

The point is not that we are not willing to help, but more about the fact that we can’t give you meaningful answers because of how different the gl-inet firmware is from the official project.

Would you want a podiatrist performing spinal surgery?

1 Like

No, I just wanted to know how this configuration would look on stock firmware. So far as I can tell gl.inet builds official firmware, and then adds a different management UI on top of it.

Apparently that's too much to ask. But whatever, I don't want to argue with you about whether or not it's supported. All I was hoping for was just to be told "here's what it would look like on the stock firmware", not to get an argument about whether or not my firmware could be supported.

I've raised it with gl.inet previously - I came here because their forum wasn't quick enough in getting me any guidance. I've now taken it directly to their support e-mail. But this has left a really bad taste in my mouth for this project.

Like I said, I'm no stranger to doing open source support. I've been doing it myself for the openSUSE project for over 20 years. If someone comes into our forums and asks about an old version or a downstream distribution, we still try to help, because there's enough knowledge around still, and enough commonality that we can help. We will advise that they should try running a current version, but we won't generally tell them to 'pound sand'.

I know that what I'm asking for is not a typical request, but I also know that the general spirit of open source communities is to provide guidance and assistance to the extent that it is possible. And in my experience, the type of question that I'm asking here isn't outside that realm of expectation. It's a firewall/routing question. Get packets from A to B.

I'm not trying to get some sort of proprietary driver working (like their multi-wan driver). I'm not asking for help fixing a CORS issue in their nginx reverse proxy that's installed on the router.

I"m asking for help with a simple routing question. That's it.

This is likely a simple network routing/configuration issue, and I simply don't know how to get the same result that NAT loopback gives me going from LAN-WAN-LAN when going from VPN-WAN-LAN.

I didn't come here looking for a fight or an argument. Really, truly, I did not. I came here because this community has been helpful to me in the past.

You don't know that your answers aren't meaningful - you're just assuming that the gl.inet firmware varies wildly from the official project (even though the build string says that it IS the official project's code and not their own fork, as far as I can see - the fact that the commit hash is valid in the openwrt source tree on github points very strongly that at least the core functionality is from the offical 21.03 build).

But you just saw "gl.inet firmware" and said "nope, sorry, can't help you there" without even considering that my question is a simple routing configuration question. You assumed that an answer about how it should have worked with iptables on that release would not be meaningful to me. Now, granted, you don't know that I've got decades of experience working with networking equipment and operating systems, and that, if pointed in the right direction, I'll probably find the answer I'm looking for.

No offer to say "hey, while you can run official code here, let's look at your zone configurations, your firewall configurations, and see if we can get you pointed in the right direction."

I'm not asking for surgery here. I'm asking for a pointer in the right direction.

But what I've learned that the people who are "leaders" and "gurus" here would rather argue about whether or not something is supported or not than just provide a simple answer to what ultimately is probably a simple question and I'm just vapor-locked on what the next step is to fix it.

And where do you draw the line about how much support should be given in any particular case? The 'rule' is in place for the simple reason that this is not a support forum for devices running firmware created by commercial entities (or for that matter anyone else who may or may not have made unknown alterations to the code).

Even if you weren't running a third-party firmware OpenWRT 21.02 has been EOL for a long time. The first piece of advice to anyone running such an old build is to update it to a current version to see if that resolves the problem. Only then is anyone likely to start troubleshooting further.

I understand that it can be frustrating when you can't get what you want but you have a simple solution to your issue. Install an official build of OpenWRT (ideally the most recent build) and then, if the issue still exists, ask for support.

1 Like

On official openwrt, it would likely just work. There could be nuance, though, as it could still depend on details of your config (in ways that may not be obvious without first testing).

That’s what I said earlier:

1 Like

Yes, but on official openwrt, what would the configuration look like? It "likely just working" is a function of defaults that are selected. So what are those defaults?

Look, I'm not here to argue about this. If you don't want to help, don't help.

Like I said, in the openSUSE forums, we get questions periodically from people running old versions, and while we advise that they upgrade, members of the community generally will still try to help - and this particular instance, as I have repeatedly said, is ultimately probably just a vapor-lock on my part at trying to remember how to configure this. A pointer in the right direction is all I need.

If you can't provide that, then, respectfully, your input isn't really helping here.

There is no “universal” config to show. We would suggest that you start by configuring the basics (install and configure wireguard, setup your port forwards, etc) and then test. And if that doesn’t work, we would evaluate your config and suggest changes based on what is/is not working. Your current config files would not work with the official Openwrt, so evaluation of those wouldn’t be useful.

1 Like

I mean, "it would likely just work" tells me that there are "sane defaults".

I suspect that my config files would work with official 21.03 (outdated as it may be). I really don't think that gl.inet would have completely rewritten everything in the openwrt code, built it to a specific release and commit hash that's in the openwrt code, only to have it not work. That seems like it would be a ton of work for them to make it incompatible with the config files in the openwrt source tree.

So, for example, I have a zone called 'wgserver' that forwards to 'wan' and 'lan'. Input/output/forward are set to "accept", "accept", and "reject", and masquerading is enabled.

In my port forwards configuration, I have port 443/tcp forwarded from the WAN to, let's say, port 3000 on my local server (192.168.8.42, let's say). Advanced settings have NAT loopback enabled.

Here are the relevant parts of /etc/config/network (with addresses obscured):

config device
	option name 'br-lan'
	option type 'bridge'
	list ports 'lan2'
	list ports 'lan3'
	list ports 'lan4'
	list ports 'lan5'

config interface 'lan'
	option device 'br-lan'
	option proto 'static'
	option ipaddr '192.168.8.1'
	option netmask '255.255.255.0'
	option ip6assign '60'
	option isolate '0'

config interface 'wan'
	option device 'eth1'
	option proto 'static'
	option force_link '0'
	option ipv6 '0'
	option metric '1'
	option ipaddr '1.2.3.4' # Provided by my ISP
	option gateway '1.2.3.5' # Provided by my ISP
	option vlanid '0'
	option netmask '255.255.255.252'
	option peerdns '0'
	option dns '127.0.0.1'

config interface 'wgserver'
	option proto 'wgserver'
	option config 'main_server'
	option disabled '0'

/etc/config/wireguard_server:

config servers 'main_server'
	option address_v4 '10.10.0.1/24'
	option address_v6 'fd00:db8:0:abc::1/64'
	option port '51820'
	option access 'ACCEPT'
	option fwmark '0x8000'
	option ipv6_enable '0'
	option client_to_client '0'
	option masq '1'
	option private_key '<privkey>'
	option public_key '<pubkey>'

(Also includes multiple peer configurations)

From /etc/config/firewall:

config redirect
        option proto 'tcp'
        option src_dport '443'
        option dest_ip '192.168.8.10' # internal server address
        option dest_port '3000'
        option src 'wan'
        option name 'GL-443'
        option dest 'lan'

If there are other configurations in stock openwrt that would be useful to look at, let me know and I'll provide them.

You may not think it's useful (and maybe it wouldn't be if I wanted you to tell me exactly what to change) to look at these, but let's just assume that they would work until such point as something in them doesn't actually make sense with stock firmware. Like I said, I'm looking for a direction to look in here, because I feel that I'm overlooking something obvious.

I have removed their nginx proxy from the VPN interface, so now what I see when I connect to the VPN and try to access the web services exposed on the WAN interface is a "connection denied". So the problem is still that it's properly resolving the IP address to the external address ("1.2.3.4") using the external DNS server, but the connection is still trying to be made to the rotuer rather than doing what effectively from the LAN interface is a NAT loopback to get to the WAN interface. It's pretty much the same as if I had NAT loopback turned off on the port forward from WAN -> LAN for 443.

No, it would absolutely not work. Your device wasn’t fully supported until 23.05.3.

Not everything, of course. But the config syntax between 21.02 and 24.10 is not compatible.