Wireguard has a pesky behavior where the UDP socket is locked to the context the interface is created in. this means that associating a WAN interface with a VRF is not possible to also associate a wireguard interface, even with ip rules. I found that the outbound handshakes will exit the VRF/table, but the replies are dropped since the UDP socket exists in the default VRF.
However, Wireguard supports namespaces, just like macvlans, where the logical interface can be moved to any namespace and the Socket or physical interface can exist in another name space. To leverage this I created a few scripts to enable route isolation between multiple WAN/untrusted connections and allow wireguard to use these isolated routes, while allowing the VPN routes to exist in the default namespace (such as using BGP)
A feature of this approach which uses macvlans to attach the physical interfaces into the namespaces allows any interface driver, including wifi station interfaces, to be used.
by using hotplug, it allows the interfaces to be controlled/reconfigured by luci/other tools simply by starting/stopping the uci interface. making it reliable and consistent
The 3 scripts operate in this way
- netns script simply reats the uci netns config to gather configured namespaces and create them
- netns-veth creates a veth pair between each namespace and the host, and sets up static routes if specified (this is necessary if we need to manage the webui of a modem, for example)
- hotplug script runs if the interfaces is listed in netns uci configuration to set it up for two roles: wan, wireguard
The hotplug script does the following for wan roles on hotplug ifup
- create a macvlan interface associated with the wan physical interface as passthru
- move this linked macvlan to the associated name space
- sets up a static ip and gateway (if configured)
- starts udhcpcd and odhcp6c (if configured)
- configures a secondary staticip (if configured, even if dhcp is configured, i found some modems that assign a public IP but use a RFC1918 address for web management, need the interface to also have this subnet configured)
- configures the namespace with a light set of hardned sysctl and firewall rules for the macvlan, including masquerading
Native openwrt uci interfaces should be configured as unmanaged with no ip address. interfaces for the default namespace (named veth-) should also be created and assigned the WAN. Port forwarding isn't supported for this use case, however would be possible by extending the script to take portforward options and configure the namespace nftables to forward them to the default namespace address of the veth pair
For wireguard interfaces, on hotplug ifup
- read the wireguard uci config for the interface
- remove the wireguard interface from the default namespace
- recreate the wireguard interface in the associated namespace
- move the wireguard interface to the default namespace
- assign the wireguard uci configured IP information
It doesn't consider setting up routes based on wireguard allowed IP, I just allow 0.0.0.0,::/0, I use BGP and so should you.
Here's an example config:
cat /etc/config/netns
config netns 'wan'
option netns 'sat'
option interface 'wan'
option type 'wan'
option hostname 'hddjejdjdbdbdj'
option broadcast '1'
option client_id '0001000123456789abcdef012345'
option dhcp '1'
option dhcp6 '1'
list routes '192.168.100.1/32'
option secondaryip '192.168.100.10/24'
config netns 'wg0'
option netns 'sat'
option interface 'wg0'
option type 'wireguard'
config netns 'wwan0'
option netns 'cell'
option interface 'wwan0'
option type 'wan'
option hostname '5fjihhbvv'
option broadcast '0'
option client_id '0001000123456789abcdef012345'
option dhcp '1'
option dhcp6 '1'
list routes '192.168.3.1/32'
config netns 'wg1'
option netns 'cell'
option interface 'wg1'
option type 'wireguard'
config netns 'trm_wwan'
option netns 'wifi'
option interface 'trm_wwan'
option type 'wan'
option hostname 'asdfasdfsf'
option broadcast '0'
option client_id '0001000123456789abcdef012345'
option dhcp '1'
option dhcp6 '1'
list routes '192.168.12.163/32'
config netns 'wg2'
option netns 'wifi'
option interface 'wg2'
option type 'wireguard'
here is the netns service
cat /etc/init.d/netns
#!/bin/sh /etc/rc.common
START=19
STOP=90
USE_PROCD=1
# Logging function
log() {
logger -t netns "$@"
}
setup_namespace() {
local cfg=$1
local netns nsname
log "Processing netns config section: $cfg"
# Get config values
config_get netns "$cfg" netns
nsname=$netns
# Create namespace if it doesn't exist
if ! ip netns list | grep -q "${nsname}-ns"; then
log "Creating namespace ${nsname}-ns"
ip netns add ${nsname}-ns
if [ $? -ne 0 ]; then
log "Failed to create namespace ${nsname}-ns"
return 1
fi
else
log "Namespace ${nsname}-ns already exists"
fi
}
start_service() {
log "Starting netns service"
# Clean up existing namespaces
stop_service
# Load netns config
log "Loading netns UCI configuration"
config_load netns
# Process each netns configuration
log "Processing netns configurations"
config_foreach setup_namespace netns
log "Netns service startup complete"
}
stop_service() {
log "Stopping netns service"
# Delete all namespaces ending with -ns
for ns in $(ip netns list | awk '{print $1}'); do
log "Processing namespace $ns"
# Only process namespaces ending with -ns
if echo "$ns" | grep -q '\-ns$'; then
# Delete namespace
log "Deleting namespace $ns"
ip netns delete "$ns"
if [ $? -eq 0 ]; then
log "Deleted namespace $ns"
else
log "Failed to delete namespace $ns"
fi
else
log "Skipping namespace $ns (does not end with -ns)"
fi
done
log "Netns service stopped"
}
restart_service() {
log "Restarting netns service"
stop_service
start_service
log "Netns service restarted"
}
service_triggers() {
procd_add_reload_trigger "netns"
}
here is the netns-veth service
cat /etc/init.d/netns-veth
#!/bin/sh /etc/rc.common
START=91
STOP=09
USE_PROCD=1
# Logging function
log() {
logger -t netns-veth "$@"
}
# Global variable to track processed namespaces
PROCESSED_NAMESPACES=""
create_veth_pair() {
local nsname=$1
local ns_ip=$2
local host_ip=$3
local veth_name="veth-${nsname}"
log "Creating veth pair for namespace ${nsname}-ns, ns_ip=$ns_ip, host_ip=$host_ip"
# Check if veth-<nsname> exists in host namespace
if ip link show "$veth_name" >/dev/null 2>&1; then
log "$veth_name already exists in host namespace, attempting to delete"
ip link set "$veth_name" down
ip link delete "$veth_name"
if [ $? -ne 0 ]; then
log "Failed to delete existing $veth_name"
return 1
fi
log "Deleted existing $veth_name"
fi
# Create veth pair
ip link add "$veth_name" type veth peer name veth-host
if [ $? -eq 0 ]; then
log "veth pair $veth_name/veth-host created successfully"
else
log "Failed to create veth pair for ${nsname}"
return 1
fi
# Move peer to namespace
ip link set veth-host netns ${nsname}-ns
if [ $? -eq 0 ]; then
log "Moved veth-host to ${nsname}-ns"
else
log "Failed to move veth-host to ${nsname}-ns"
return 1
fi
# Configure host side
ip addr add ${host_ip}/30 dev "$veth_name"
ip link set "$veth_name" up
if [ $? -eq 0 ]; then
log "Configured $veth_name with ${host_ip}/30 and set up"
else
log "Failed to configure $veth_name"
return 1
fi
# Configure namespace side
ip netns exec ${nsname}-ns ip addr add ${ns_ip}/30 dev veth-host
ip netns exec ${nsname}-ns ip link set veth-host up
if [ $? -eq 0 ]; then
log "Configured veth-host in ${nsname}-ns with ${ns_ip}/30 and set up"
else
log "Failed to configure veth-host in ${nsname}-ns"
return 1
fi
}
setup_veth_and_routes() {
local cfg=$1
local netns routes nsname
log "Processing netns config section: $cfg"
# Get config values
config_get netns "$cfg" netns
nsname=$netns
# Check if namespace has already been processed
if echo "$PROCESSED_NAMESPACES" | grep -qw "$nsname"; then
log "Warning: Namespace $nsname already processed, skipping section $cfg (possible UCI configuration error)"
return 0
fi
# Get routes for this specific section
routes=$(uci -q get netns."$cfg".routes | tr '\n' ' ' | sed 's/[[:space:]]*$//')
log "Config values: netns=$netns, routes='$routes'"
# Skip if no routes are defined
if [ -z "$routes" ]; then
log "No routes defined for $nsname in section $cfg, skipping veth creation"
return 0
fi
# Mark namespace as processed
PROCESSED_NAMESPACES="$PROCESSED_NAMESPACES $nsname"
# Check if namespace exists
if ! ip netns list | grep -q "${nsname}-ns"; then
log "Namespace ${nsname}-ns does not exist, skipping"
return 1
fi
# Calculate unique IPs in 169.254.0.0/16 space for /30 subnets
local idx=$(ip netns list | grep -n "${nsname}-ns" | cut -d: -f1)
# Each /30 subnet uses 4 IPs, so calculate the subnet base
local subnet_base=$(( (idx - 1) * 4 ))
# Calculate the second and third IPs (usable hosts) in the /30 subnet
local ns_ip_octet=$((subnet_base % 256))
local ns_ip_third_octet=$((subnet_base / 256))
local ns_ip="169.254.$ns_ip_third_octet.$((ns_ip_octet + 1))" # Second IP
local host_ip="169.254.$ns_ip_third_octet.$((ns_ip_octet + 2))" # Third IP
log "Calculated IPs: ns_ip=$ns_ip, host_ip=$host_ip, idx=$idx, subnet_base=$subnet_base"
# Create veth pair
create_veth_pair "$nsname" "$ns_ip" "$host_ip" || return 1
# Verify veth is up
if ip link show "veth-${nsname}" | grep -q "state UP"; then
log "veth-${nsname} is up and ready for routing"
else
log "veth-${nsname} is not up, route addition may fail"
fi
# Add routes to default namespace, explicitly associating with veth-${nsname}
for route in $routes; do
local veth_name="veth-${nsname}"
log "Adding route $route via $ns_ip dev $veth_name in default namespace"
# Check if route already exists
if ip route show "$route" | grep -q "via $ns_ip dev $veth_name"; then
log "Route $route via $ns_ip dev $veth_name already exists, skipping"
continue
fi
ip route add "$route" via "$ns_ip" dev "$veth_name"
if [ $? -eq 0 ]; then
log "Route $route added successfully"
else
log "Failed to add route $route: $(ip route add "$route" via "$ns_ip" dev "$veth_name" 2>&1)"
fi
# Log current routing table for debugging
log "Current routing table: $(ip route show | grep "$route")"
done
}
start_service() {
log "Starting netns-veth service"
# Reset processed namespaces
PROCESSED_NAMESPACES=""
# Clean up existing veth interfaces and routes
stop_service
# Load netns config
log "Loading netns UCI configuration"
config_load netns
# Process each netns configuration
log "Processing netns-veth configurations"
config_foreach setup_veth_and_routes netns
log "Netns-veth service startup complete"
}
stop_service() {
log "Stopping netns-veth service"
# Load netns config to get namespaces defined in UCI
log "Loading netns UCI configuration"
config_load netns
# Process each netns configuration to clean up associated veth interfaces and routes
cleanup_veth_and_routes() {
local cfg=$1
local netns nsname
# Get config values
config_get netns "$cfg" netns
nsname=$netns
local veth_name="veth-${nsname}"
log "Cleaning up veth and routes for namespace ${nsname}-ns"
# Check if veth interface exists in host namespace
if ip link show "$veth_name" >/dev/null 2>&1; then
# Remove routes pointing to the veth interface
log "Removing routes for $veth_name"
ip route show | grep "dev $veth_name" | while read -r route; do
local route_dest=$(echo "$route" | awk '{print $1}')
ip route del "$route_dest"
if [ $? -eq 0 ]; then
log "Deleted route for $veth_name: $route_dest"
else
log "Failed to delete route for $veth_name: $route_dest"
fi
done
# Bring down the veth interface
log "Bringing down $veth_name"
ip link set "$veth_name" down
if [ $? -eq 0 ]; then
log "$veth_name set down"
else
log "Failed to set $veth_name down"
fi
# Delete veth interface
log "Deleting $veth_name in default namespace"
ip link delete "$veth_name"
if [ $? -eq 0 ]; then
log "Deleted $veth_name"
else
log "Failed to delete $veth_name"
fi
else
log "No $veth_name found in default namespace"
fi
# Check if namespace exists and clean up veth-host
if ip netns list | grep -q "${nsname}-ns"; then
if ip netns exec "${nsname}-ns" ip link show veth-host >/dev/null 2>&1; then
log "Found veth-host in ${nsname}-ns, deleting"
ip netns exec "${nsname}-ns" ip link set veth-host down
ip netns exec "${nsname}-ns" ip link delete veth-host
if [ $? -eq 0 ]; then
log "Deleted veth-host in ${nsname}-ns"
else
log "Failed to delete veth-host in ${nsname}-ns"
fi
else
log "No veth-host found in ${nsname}-ns"
fi
else
log "Namespace ${nsname}-ns does not exist, skipping veth-host cleanup"
fi
}
# Process each netns configuration
log "Processing netns configurations for cleanup"
config_foreach cleanup_veth_and_routes netns
log "Netns-veth service stopped"
}
restart_service() {
log "Restarting netns-veth service"
stop_service
start_service
log "Netns-veth service restarted"
}
service_triggers() {
procd_add_reload_trigger "netns"
}
eventually, I will work on an openwrt package to extend uci/luci to enable namespaces for interface and firewall configuration, so it could possibly allow injecting "ip netns exec " infront of the commands if the advanced "netns" feature is configured. as well as specify the "create netns" and "destination netns" for interfaces. But for now, these do exactly what I need.