I believe there is some issue with PBR when the traffic originates from the router itself. I can recall a couple of more topics with the same problem. But if you managed it with off router WG server and it works fine for you, I am happy.
Feel free to mark the topic as solved.