IFBs in WireGuard Context

Installing and Using OpenWrt
1

What is the correct way to mirror packets from a WireGuard interface using tc commands?

With the following:

	# ifb interface for handling ingress on WAN (and VPN interface if wg show reports endpoint)
	ip link add name ifb-wg-pbr type ifb
    ip link set ifb-wg-pbr up

	# capture all packets on WAN
    tc qdisc add dev $wan_if handle ffff: ingress
    tc filter add dev $wan_if parent ffff: prio 2 matchall action mirred egress redirect dev ifb-wg-pbr

	# if 'wg show' reports an endpoint, then pass over wireguard packets on WAN and capture all VPN packets
	if [ ! -z "$wg_endpoint" ]
	then
       		tc filter add dev $wan_if parent ffff: protocol ip prio 1 u32 match ip src ${wg_endpoint}/32 action pass
       		tc qdisc add dev $vpn_if handle ffff: ingress
       		tc filter add dev $vpn_if parent ffff: matchall action mirred egress redirect dev ifb-wg-pbr
	fi

I see some apparently mangled packets on the ifb using 'tcpdump -i ifb-wg-pbr' like:

11:02:12.674405 40:00:6c:06:86:f4 (oui Unknown) > 45:00:00:57:d1:22 (oui Unknown), ethertype Unknown (0x3470), length 87:
        0x0000:  7813 0a05 0002 01bb ce0b 96af 9437 464f  x............7FO
        0x0010:  8843 5018 07fb 2b58 0000 1703 0300 2a00  .CP...+X......*.
        0x0020:  0000 0000 0000 b390 2546 fbc5 aeef 7af5  ........%F....z.
        0x0030:  5c6d 549d bf3a da05 825c 5a16 8dd9 0905  \mT..:...\Z.....
        0x0040:  6a67 24dd 40ee 822a aa                   jg$.@..*.

To me this suggests that:

tc qdisc add dev $vpn_if handle ffff: ingress
tc filter add dev $vpn_if parent ffff: matchall action mirred egress redirect dev ifb-wg-pbr

is not working in the same way that it does with wan, which gives values like:

11:02:12.734113 IP one.one.one.one > 10.217.XX.YY: ICMP echo reply, id 62949, seq 3623, length 64
11:02:12.776169 IP one.one.one.one > 10.217.XX.YY: ICMP echo reply, id 6932, seq 3622, length 64

What am I missing? Any ideas @dave14305? Is the problem that ingress packets in the vpn interface are encrypted? But if I look at the vpn interface using tcpdump -i vpn I see everything unencrypted:

11:13:16.882407 IP 10.5.0.2.53269 > 13.107.138.9.443: Flags [P.], seq 1744:1995, ack 1, win 1024, length 251
11:13:16.931266 IP 13.107.138.9.443 > 10.5.0.2.53269: Flags [.], ack 1380, win 16387, length 0

So am I wrong in thinking that the mirroring will work properly? Is it not possible to mirror ingress on a WireGuard interface to end up with the inner packets to give to IFB for CAKE?

Actually @tohojo I am going to tag you because this seems so specialized. My ultimate end goal is just to get separate IFBs correctly set up for both ingress and egress on my WAN and VPN interfaces so that I can get qosify to DSCP mark before CAKE is applied. I wrote this script to handle this situation, and I thought it worked, but now I am not sure.

2

No, the problem is that a wireguard interface is layer 3 only, so the packets are missing an Ethernet header. Which means that just redirecting won't work. You could do the redirecting with a BPF program and have that add an Ethernet header first (and remove it again later), but that becomes messy quickly...

1 Like
3

Thanks that saves a lot of trouble. But I suppose I can construct my two upload/download IFBs from ingress and egress br-lan and br-guest? Skip over LAN-LAN traffic?

That'll work right?

4

No idea how redirecting to IFB interacts with bridge interfaces - try it and see? :slight_smile:

5

Here is my first attempt - it seems to work:

	# ifb interface for handling ingress on WAN (and VPN interface if wg show reports endpoint)
	ip link add name ifb-ul type ifb
	ip link add name ifb-dl type ifb
    ip link set ifb-ul up
    ip link set ifb-dl up

    tc qdisc add dev br-lan handle ffff: ingress
    tc qdisc add dev br-guest handle ffff: ingress
	tc qdisc add dev br-lan handle 1: root prio
	tc qdisc add dev br-guest handle 1: root prio

	# capture upload (ingress) on br-lan and br-guest
	tc filter add dev br-lan parent ffff: protocol ip prio 1 u32 match ip dst 192.168.1.0/24 action pass
	tc filter add dev br-lan parent ffff: prio 2 matchall action mirred egress redirect dev ifb-ul
	tc filter add dev br-guest parent ffff: protocol ip prio 1 u32 match ip dst 192.168.2.0/24 action pass
	tc filter add dev br-guest parent ffff: prio 2 matchall action mirred egress redirect dev ifb-ul
       	
	# capture download (egress) on br-lan and br-guest
	tc filter add dev br-lan parent 1: protocol ip prio 1 u32 match ip src 192.168.1.0/24 action pass
	tc filter add dev br-lan parent 1: prio 2 matchall action mirred egress redirect dev ifb-dl
	tc filter add dev br-guest parent 1: protocol ip prio 1 u32 match ip src 192.168.2.0/24 action pass
	tc filter add dev br-guest parent 1: prio 2 matchall action mirred egress redirect dev ifb-dl

	# apply CAKE on the ifbs
	tc qdisc add dev ifb-ul root cake bandwidth 30Mbit diffserv4 triple-isolate nonat nowash no-ack-filter split-gso rtt 100ms noatm overhead 92
    tc qdisc add dev ifb-dl root cake bandwidth 25Mbit diffserv4 triple-isolate nonat nowash ingress no-ack-filter split-gso rtt 100ms noatm overhead 92

The nice thing about this approach is that it will work with lots of different configurations.

6

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.