My current network setup is PPPoE-WAN and then Wireguard as the default route - VPN Policy Routing as needed for specific IPs (via TCP by way of ports 80 and 443). Unbound working as a recursive resolver is the DNS solution serving the entire network. Unbound uses exclusively the Wireguard interface for its outgoing traffic. To that end, I've figured that the following rules work if I prefer to manually set the MSS to Wireguard's default "1380" (I believe that's correct given my grasp of packet/header sizes and MSS functionality; Wireguard's packet overhead gleaned here):
iptables -t mangle -I FORWARD -o wgclient -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1380
iptables -t mangle -I FORWARD -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1380
iptables -t mangle -I OUTPUT -o wgclient -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1380
iptables -t mangle -I OUTPUT -o pppoe-wan -p tcp -m tcp --tcp-flags SYN,RST SYN -j TCPMSS --set-mss 1380
My logic for the current ruleset is:
- FORWARD rules are instated in the mangle table by default when using the pre-configured MSS-Clamping Fix
- Setting MSS manually to 1380 for both makes sense in the context of Unbound doing recursion exclusively via Wireguard; even if it's inefficient for the few domains which are resolved and served to a client via WAN it's irrelevant.
- I tacked on the OUTPUT chain rules because I figured that's how the outgoing/outbound traffic will head during recursion - hence it would be beneficial to also clamp MSS on it. It seemed pointless if traffic between zones was clamped, but if my local recursion was attempting to shift packets outward with an incorrect MSS set (or not an explicit one, I guess?) then that adds potentiality of inefficiency/delays/SERVFAIL response.
- I can't quite recall where, but I came across a comment where somebody said that with conntrack rules enabled the MSS clamping rules should be set before them, otherwise SYN-ACKS won't also be clamped (something like that). Therefore I deigned to use the insert delegation (-I) rather than append (-A).
- My network is currently IPv4 exclusively (used imagebuilder to compile a build with all IPv6 packages removed and used the disable IPV6 argument set). As stated in the link above, an exclusively IPv4 setup would allow for a potential maximum MTU of 1440 on Wireguard (or 1432 with PPPoE), so 1420 as the set MTU and, consequently, an MSS of 1380 would be appropriate - rather than something lower.
What I'm wondering with all this is, how does this all track within the context of my desired operability and, ultimately, is it the most optimal? Should I lay out the rules differently - using different chains (POSTROUTING, PREROUTING), or anything else? Thanks.