She accesses the site like anyone else, types in the domain. i.e. DNS gives her my PUBLIC IP and off it should go. Again the wireguard is NOT set to tunnel web traffic so there is no way (that I can find) that this should be screwing up web.
Does this method work for her on her phone, for example?
Method? You mean loading my sites? Sure it works on her computer too if she turns Wireguard off. I said all that in the initial post. If she wants the web stuff she has to kill the VPN, if she wants file servers she has to enable the VPN but then loses the web.
So Wireguard is implicated...
Have you tried it on your phone (while on cellular) with wireguard enabled vs disabled?
I do not own a phone.
Yus, it's screwing something up but again when you load any other website and it works...why JUST mine. I had thoughts perhaps because the WG endpoint is the same as the domain endpoint but this is stupid. If this was the reason it would mean ports don't matter and no IP can have more than one thing available. i.e. WG isn't on 80/443 so why would that cause the issue? Like wise my web server isn't listening on 51820. OpenWRT is forwarding accordingly it shouldn't "magically" mix traffic for the two services...all my other services like for XMPP work...so WHAT IS GOING ON!!!??? (tears imaginary hair out)
Does your daughter have a phone? Does she have wireguard on the phone? It would be good to see the comparison of wg on/off and if it affects the ability to access the website on a different system than her computer.
She has a spybrick yes, it does not have wireguard access. Her machine should be fine, it's a fresh install. Like just a few weeks if that.
Well, I don't know what else to tell you since you don't have other devices to use for testing to see if the wireguard issue is unique to a specific device or if it's consistently causing issues.
My guess is that the problem is with your daughter's computer and that different system (i.e. different OS) might prove (or refute) that assumption. But for that to work, there needs to be another device that can operate on a different network than yours (i.e. cellular, or anywhere other than your home) with Wireguard installed and connected back to your home with the same general configuration (obviously different keys and peer address). Then comparing the WG enabled/disabled behavior can give us another datapoint.
Without that, we can only guess. And troubleshooting your daughter's computer is out of scope for this forum because that's not a running/behind OpenWrt.
Well the thing to me is single device or many devices the issue is still all the tools and testing say "this should work" but then it doesn't.
I just redid that machine like a month ago and outside the openWRT device EVERY machine is Arch Linux so I'm not sure what "other os" you're referencing.
Yeah but this is a resource and time constraint. If another device works, great, but it doesn't help. More so if this other datapoint was a cell phone the "OK so what's different" becomes an explosion of possibilities that are simply too many to even entertain as a reference/comparison.
And that is what I've been doing for weeks. Again nothing wrong with her machine. Every machine is running Arch, wireguard installed from the same repo, you've seen the config...wireguard technically works...
I will keep slamming my head on this but thanks for your time either way. I will have to dig in with something like wireshark eventually and try to see where/why everything works but the damned web browser...and no it's not "A" browser, it's all browsers right down to w3m. heh.
One thing I want to try if I'm going to install firejail and force the nic (or write a wrapper to run the browser in a separate namespace) and see if that works. That will at least side step the host file hackery for now.
There can be reasons -- beyond that of the OpenWrt configuration -- that a system may not work as expected.
For example (not applicable here, but a point of reference)... the default Windows firewall will not accept connections from a foreign subnet. So if you tried to connect to a Windows machine via Wireguard (where Wireguard is running on the router and not masqueraded), it will not work.
I think you're missing the point here... the idea is that it can save time by helping to identify the scope of the problem. Right now, we don't know what is actually causing the problem. It shouldn't be related to the Wireguard configuration (based on what we see), but in order to really prove that this assertion is true, we need a 'control' device that runs a different OS. For example, if you were to run an essentially identical WireGuard configuration on a Mac and it did not experience the issue, it means you need to focus on the Arch configuration for troubleshooting. If the same experience occurs on the Mac, the common denominator is the WireGuard config and thus that then becomes the focus.
In summary, a set of controlled experiments with different platforms can help narrow down the potential causes of the problem.
Yes, WG is working, but accessing the web page from doesn't for some reason -- we don't know, and cannot currently determine if it's some obscure setting/config in Arch (or a behavior/quirk of Arch in general), or if it is actually a problem with your WireGuard/OpenWrt configs.
No I get that but the scope is also at the same time 100%. If I had 50 family members using this and 1 had an issue OK but being able to tunnel back and forth with my daughter/grand daughter is kinda the whole show for me. If she had another device that worked it doesn't help unless that device could/would replace the problematic one until a solution was/is found. Her phone for example working is useless other than to say "it's just the laptop." Which lands me in the exact same place...WHAT is wrong with THAT device. If another device has the same issue, WHAT is wrong with THOSE devices.
If I had 10 devices all with the same problem then as datapoints that would look more like openwrt's wireguard but at present I suspect WG more than the machine because I'm less familiar with openwrt, lots of details I don't know in how tools are stripped and things are changed due to memory concerns, flash storage, etc. Toss in I haven't really thought about routing since the 90's back when I'd write rules by hand for BSD. I was pretty OK with those days being distant nightmares. Then I also had some issues with the newest flash which I touch on a bit further in this message.
Given I've run Arch (or Linux in general) for well over 20 years I'm going to go with there is no odd quirk. More so because I just did that machine. It's not like it was some OEM Windows install with god knows what bloatware and trash that may be causing issues. Everything on the machine was done so by me. I don't do "DE" Distros, I use base arch and build up rather than meta packages.
Does that mean I could have missed something, hell no. However I've been over the settings back and forth for weeks now and can't find a damn thing wrong. Like I said, dig, ping, tracepath, everything resolves the domain fine with Wireguard enabled. I also know this isn't strictly an openwrt issue, just I figured someone here may know what the issue was.
Honestly this is what I think is it. I say this because #1 I know her machine infinitely better than I do openwrt (despite it being Linux). I also know some things got weridly screwed up when I flashed to the newest build of openwrt. I had to relearn several things, completely redo wireguard, readd a few settings that changed and diagnose some odd behaviors.
I have some ideas of what I can do to continue to try to track down where in the chain things fail but I'm locked out for the moment. I suspect the machine went to sleep and her day has been non-stop work/school/family chaos and I won't likely see the machine come back online until tomorrow.
Either way again thank you for your time. I might seem grumpy and I am but it's not with you. This is all very frustrating, more so when all the tools you use you diagnose this stuff say "Yup, it's good!" /me runs from the room screaming! ![]()
I wholeheartedly disagree... yes, I get that an alternate device doesn't solve the problem on the device that is not working, but it does direct the troubleshooting (in other words, maybe you should be asking on Arch forums).
But, seeing as you are both unwilling and unable to use some alternate device/OS, it's all a moot point.
All of your devices are using the same platform -- Arch. I'd recommend turning your attention to that platform as the likely culprit that needs resolution. Why would I make this suggestion?? Because the WG configuration does not look like it could interfere with the web access, and absent any information to refute that assessment, the only common factor is your arch systems.
How long have you been using WireGuard on Arch? And how do you know that the behavior would be the same on some other Linux flavor (say Ubuntu or Debian)? We have seen odd behaviors from Mint, as an example.... just because you've been using linux for years doesn't mean it doesn't have some quirks to it.
Yes and no.... there are differences between each Linux distro... sometimes things don't work the same way. A good example here is the build environment for OpenWrt -- it is written based on a few specific distros. Users who try to build on other flavors sometimes run into issues with dependencies or config details that are indeed different.
Just because you did it doesn't mean there couldn't be an issue. That issue could be something you did or did not do, or it could be a bug or a quirk of the system. Understand that I'm not disputing your knowledge of Linux, but rather that it is clearly a confluence of factors that are causing the issue and it is impossible to rule out those machines as problematic.
Are you suggesting that there is no way that anything could be wrong on those machines??
Have you tried using WireShark and running pcaps with and without the WG interface running on the Arch box as well as your OpenWrt router? Maybe you can find out where the packets are going.
I agree. In fact, I would go so far to say that this is not an OpenWrt issue at all. At least until it can be proven otherwise (which requires tests on other platforms, at least another linux distro such as Ubuntu).
It's not OpenWrt.... it might be WireGuard on your Arch systems.
I do understand that this is frustrating... I've dealt with similarly frustrating problems both at home and at work. As an engineer, I make a point to run a series of controlled experiments with the intent of finding the specific root causes, carefully controlling all of the variables along the way.
I apologize if I sound at all grumpy here, too. I'm honestly trying to help, but when the experiments we need to help identify the problem are not possible to run, it becomes just as frustrating for me (in that I can't help you find a solution) as the actual problem is for you.
That said, my recommendations are to examine the behavior of Arch in more detail, including pcaps on that machine to try to figure out if the traffic is going out via the tunnel, ethernet, or being dropped and/or if the return traffic is coming back through those interfaces but getting dropped by the kernel. Likewise on OpenWrt, those pcaps can help you figure out if the traffic is actually getting to your router in the first place (when WG is enabled on your daughter's Arch machine).
There is another topic strongly in favour of testing -specifically- a phone, namely to detach yourself from the subnet overlap issue (but using the cellular data/ IP instead (which is likely to be out of the 10.0.0.0/8 subnet, another potential clash to keep in mind)). This is an important and very-very useful step, especially with your 'suboptimal' subnetting in the loop.
in other words, maybe you should be asking on Arch forums
Because this is a networking issue not an OS issue. If anything it would be more fitting to ask on a dedicated Wireguard forum. Alternatively how NetworkManager handles the Wireguard interface would make more sense.
unwilling and unable to use some alternate device/OS
"Other" OS's are not free.
I'm not sure why you keep addressing "Arch" as some strange "thing." It's the same OS that runs the Steam Deck ;p. Again Arch is not the issue, the wireguard client I will 100% concede could be. Also "Arch" does not mean the same Network system is being used. Some machines use Systemd-Networkd some are using NetworkManager. Saying the issue is "Arch" is like saying all democracies are hopeless because "Look at the US" ![]()
How long have you been using WireGuard on Arch?
I have NEVER used Wireguard. I used OpenVPN. I just switched to it about 2 months ago and have been fighting it ever since. Then I reflashed my router and had to redo it.
there are differences between each Linux distro
Yes, but I'm dealing with one distro, one I've ran for years with more than one network backend.
it is impossible to rule out those machines as problematic.
I said in a previous post I could very likely have missed something but after weeks of digging I can find nothing. The only aspect of this I DO NOT have decent knowlege of is Wiregaurd and how it interacts with openwrt.
Are you suggesting that there is no way that anything could be wrong on those machines??
No, in fact just the opposite. It should be "couldn't" not "could."
Have you tried using WireShark
No but that is on the to-do try list. The problem is the machine is not always up for me to blast with tests so it's check, fiddle, wait wait wait...
Maybe you can find out where the packets are going.
That is the hope, again every other trace thing I've said resolves correctly. Nothing I've tested shows any wrong router, IP, interface, nothing. Just fire up the browser, point it at the domain and watch it immediately say "NOPE!"
I agree. In fact, I would go so far to say that this is not an OpenWrt issue at all.
This is the point of me saying that. I had hoped someone might know what the issue was here, not that it IS an openwrt issue.
It's not OpenWrt.... it might be WireGuard on your Arch systems.
Yes it could be the wireguard client. Again first time using it, can't say I like it. That said I couldn't get openVPN to work at all on openwrt, that's why I switched.
As an engineer, I make a point to run a series of controlled experiments with the intent of finding the specific root causes
I hear that but again, that is a resource constraint. I don't have 10 years of MacBooks or Windows machine laying around, I thew all that out long long ago. However without these devices being outside the network on top of "having these devices" it's just not doable. Which is also to say I could spin up debian machines but there is no way to put them on the other side of my network. Testing INSIDE everything works fine.
I apologize if I sound at all grumpy here, too. I'm honestly trying to help
I know you are, you've helped me many times but that is also why I didn't come running here in the first 10 minutes. I try to read, test and do everything I can before posting. However without outside devices no matter the hardware or OS I have a sample size of 1 which accounts for 100% of the issue.
At some point I may have to take one of my laptops to the Library and see what I can see.
to try to figure out if the traffic is going out via the tunnel
ip route show says it's going out the normal interface, NIC => ISP router/modem and the address class that should be resolved should not go out the 10.0.0.x interface for WG. There is no DNS on the WG side for clients either...perhaps I should jerry rig my openwrt to port forward port 53 requests from the 10 class address to hit my DNS and see if it works either way.
subnet overlap issue
This again is confusing, the A class 10.x.x.x is only on the wireguard interface. The subnets that overlap should never be seen. i.e. her machine will never see my 192 subnet and the openwrt router will never see her 192 subnet. Also why do local subnets still factor into how a public IP isn't being resolved? Though that statement isn't 100% because it resolves in so far as wireguard will connect to it.
Because this is a networking issue not an OS issue
Yes, but how the OS handles networking can be different.
If anything it would be more fitting to ask on a dedicated Wireguard forum.
No harm in that.
Alternatively how NetworkManager handles the Wireguard interface would make more sense.
That is deep within the OS... so I consider it an OS related item, but yeah, that too.
"Other" OS's are not free.
Ubuntu is totally free. As is Debian.
You may not own a phone, but you probably have a friend who does and who you could trust to help you with this (you'd just need to give them a Wireguard config, you can revoke it after the test is done).
I'm not sure why you keep addressing "Arch" as some strange "thing." It's the same OS that runs the Steam Deck ;p. Again Arch is not the issue, the wireguard client I will 100% concede could be. Also "Arch" does not mean the same Network system is being used. Some machines use Systemd-Networkd some are using NetworkManager. Saying the issue is "Arch" is like saying all democracies are hopeless because "Look at the US"
Arch is a specific distribution of Linux. There are dozens of popular ones. In fact, even MacOS is based on Unix (but yes, that's not 'free' in that you need a Mac). Using anything other than your current Arch installation can be useful as a troubleshooting datapoint.
I have NEVER used Wireguard.
So you don't have experience with how WireGuard interacts with Arch linux (or any system).
The only aspect of this I DO NOT have decent knowlege of is Wiregaurd and how it interacts with openwrt.
You also don't have knowledge of how Wireguard interacts with Arch. By definition if you've never used WireGuard.
This is almost certainly not an OpenWrt issue.
That is the hope, again every other trace thing I've said resolves correctly.
Resolving is not the same as where the packets get routed. That's like saying that I can identify Madrid on the map, but it doesn't tell me how to get there from my home.
I hear that but again, that is a resource constraint. I don't have 10 years of MacBooks or Windows machine laying around, I thew all that out long long ago.
All you need is a PC that can run linux (I'd recommend Ubuntu, it's free) and can be placed on a network outside your home and ideally not colliding with your subnets.
Or a phone that belongs to a friend -- could be at your home or theirs or at a cafe... doesn't matter as long as it can be on cellular.
Which is also to say I could spin up debian machines but there is no way to put them on the other side of my network. Testing INSIDE everything works fine.
If you have a laptop of any kind, you can take it to a cafe or a friend's house.
Testing inside doesn't tell you anything useful in terms of resolving this issue.
At some point I may have to take one of my laptops to the Library and see what I can see.
Now we're talking!
ip route show says it's going out the normal interface, NIC => ISP router/modem and the address class that should be resolved should not go out the 10.0.0.x interface for WG. There is no DNS on the WG side for clients either...perhaps I should jerry rig my openwrt to port forward port 53 requests from the 10 class address to hit my DNS and see if it works either way.
pcaps will tell more than traceroute.
No harm in that.
No but I'm not part of one so I asked here. Again I wasn't saying this is an openwrt issue, just hoping someone here (because it is to a degree a networking forum) would have encountered something like this with their homelab and been able to say oh yeah this is X, set this and this and it will stop being stupid.
Ubuntu is totally free. As is Debian.
Well when you said other os's you made it sound like "How DARE you not have some $4000 Macbook laying around!"
Debian I can do, I can do any distro, but again, it will be INSIDE the network so not a good test. Also moving to a debian machine is more of a wireguard client test given debian is also systemd so systemd-networkd and or NetworkManager.
You may not own a phone, but you probably have a friend who does and who you could trust to help you with this (you'd just need to give them a Wireguard config, you can revoke it after the test is done).
I could not impress upon you how technically illiterate those in my life are. There is no way I could recuite help this way. I will drag my ass to the library with a laptop in a day or two if I can't make headway. Right now everything is DOA until my daughter powers the machine back on.
MacOS is based on Unix
Yes it is a blend of Open and Net BSD. I've been doing this crap since the 80's. It's why I'm grumpy and fed up.
("This crap" i.e. tech, not networking, I HATE networking.)
So you don't have experience with how WireGuard interacts with Arch linux (or any system).
Nope which is why I said I know her machine more than openwrt or Wireguard. I know how Arch is set up and how I set it up. What I DO NOT know is what under the hood changes openwrt employs when stripping tools and various tricks used to operate on such small memory foot prints and such. Nor am I familiar with Wireguard server or client. It's been months of reading guides that make it sound like magic but it's been an obnoxious uphill battle. However I don't know if this is because of Wireguard or because it's server is run on such a constrained platform. Wiregaurd is all new and I'd kinda hate it.
This is almost certainly not an OpenWrt issue.
Heh again never said it was.
Resolving is not the same as where the packets get routed. That's like saying that I can identify Madrid on the map, but it doesn't tell me how to get there from my home.
Yeah but this comes back to OK so then who (as in what tools) will give me this picture. "Wireshart" is clearly on the menu but I've not got there yet.
I'd recommend Ubuntu, it's free
All BSD/Linux is free but I'd NOT use Ubuntu, Canonical has really pooched it. Pure Debian if I was going to run Debian. I do technically have a few VOID machines but again, all IN network. (VOID for the systemd hater in me)
If you have a laptop of any kind, you can take it to a cafe or a friend's house.
Like I said I'll have to hit the library.
Testing inside doesn't tell you anything useful in terms of resolving this issue.
I DUN SAID THAT.
( I really wonder some days if my "accents" and snark is seen as hateful or if the jokes land. )
pcaps will tell more than traceroute.
I've got a running list. I'm just impatiently waiting for her to show up to power the damned thing back on. I need to write a little script to block power management while I'm connected.
I just had another "wtf" in trying to think this through about the address classes. I mean if someone is working from a hotel and the local hotel uses the same subnet this kinda makes wireguard trash if various network conditions can randomly screw over your access. Expecting a client to ALWAYS have a unique subnet is really some wishful thinking...
That is correct and that is why using a subnet which is much used e.g. 192.168.0.x and 192.168.1.x is sub optimal
From my notes: WireGuard Server Setup Guide
Check all involved subnets
As WireGuard is a routed solution all three involved subnets have to be different and non overlapping. So the routers subnet, the WireGuard subnet and the Clients subnet all have to be different.
As you often cannot choose the subnet of the WireGuard client, it is best to avoid using frequently used subnets for your router e.g. 192.168.1.1/24 or 192.168.0.1/24 but if you cannot easily change the routers subnet then leave it and hope for the best.
For the WireGuard subnet 172.22.22.1/24 is chosen in this example which is not often used.
All (routed) VPN solutions have the same problem so it is not exclusively for WireGuard.
About connecting to your daughters PC not only do you have the subnet problem but a standard WireGuard client is designed with connecting to a VPN provider in mind so it will only allow outgoing traffic otherwise your VPN provider (and its users) might get into your PC.
So you also have to allow incoming WireGuard traffic by tweaking the firewall.
So you also have to allow incoming WireGuard traffic by tweaking the firewall.
Not sure what you mean by this given Wireguard clients are connected i.e. this is already set.
not only do you have the subnet problem
Again neither side of the connection ever see the overlapped addresses. In addition to this I have everything functional EXCEPT that when wireguard is enabled on the client web traffic to the wireguard server does not work. So again a completely different IP class. Everyone is focused on the things I have working, not the problem I posted about.
WireGuard client is designed with connecting to a VPN provider
Yes but I am the provider heh. Lots of the things I read when setting things up were constantly talking about 3rd party setups which was rather unhelpful.
Either way I'll close this and continue to smash my head on this.
Yes but I am the provider heh
Yes but the standard client does not know that.
For bi-directional traffic you use a site-to-site setup but for setting this up using two routers is the way to go, all in my notes.
Not that it is impossible to do an a PC but as said you have to allow incoming traffic on the WireGuard interface by tweaking the firewall.
If the basic setup is wrong as in your case you have to revert to masquerading but in the end it will bite you as you have found out.