Iptables TTL mangling doesn't work with bridged setup

[wh2k9]

I have a GL.iNet AR-300M running the latest OpenWRT 18.06.2 stable release. I'm using it basically as a USB-ethernet bridge between my USB-tethered Android phone and my primary router (a pfSense box) sitting upstream. In order to minimize unnecessary layers of NAT I've set up a bridge in OpenWRT (br-lan) that consists of one of the ethernet ports (eth0) and the USB tether interface, usb0.

Because my cell carrier imposes tethering restrictions (enforced by checking the outgoing TTL of packets from the phone) I've configured iptables to modify the TTL as follows:

iptables -t mangle -I POSTROUTING -m physdev --physdev-out usb0 -j TTL --ttl-set 65

However, this doesn't seem to work. The firewall rule shows up in Status -> Firewall but the packet counters remain at 0, indicating that this mangle rule isn't been applied (confirmed with packet sniffing on the phone).

If I tweak OpenWRT to use a standard routed NAT setup, the mangling works with the following iptables rule:

iptables -t mangle -I POSTROUTING -o usb0 -j TTL --ttl-set 65

However, this isn't ideal as this creates three layers of NAT before traffic leaves the phone (pfSense router, OpenWRT router, Android tether).

Anyone have any ideas as to why mangling doesn't seem to work with a bridged interface? I've already checked that the appropriate packages are installed (iptables-mod-physdev, iptables-mod-ipopt).

[jeff]

iptables typically works with routed packets, and doesn't "see" bridged packets. I believe you'll need ebtables to work with bridged packets.

[wh2k9]

Correct me if I'm wrong, but isn't the iptables physdev module specifically designed to work with bridged traffic/bridged interfaces?

http://ipset.netfilter.org/iptables-extensions.man.html#lbBQ

[lleachii]

• Have you installed this module?
• Are you saying it isn't working?

https://openwrt.org/packages/pkgdata/kmod-ipt-physdev

opkg update
opkg install kmod-ipt-physdev

[wh2k9]

Yep, it's installed. In fact kmod-ipt-physdev is a dependency of the iptables-mod-physdev package.

[wh2k9]

Well, I'll be darned. I did a reset to stock settings and redid the configuration from scratch. It's working now! Not sure what was going wrong - perhaps the br_netfilter kernel module wasn't loaded.

[wh2k9]

Nope, not working again. It seems that after a power cycle the rule stops working.

[vgaetera]

@wh2k9

It seems that after a power cycle the rule stops working.

Are you sure that your rule is applied properly on firewall service reload/restart and system reboot?

[lleachii]

@wh2k9, you have to properly add those rules into the UCI, LuCI at Firewall > Custom Rules, or to /etc/firewall.user - to be persistent on reboot.

[wh2k9]

Yep, that's how I originally added the rule in the first place

I did some more testing. The bridge + physdev method seems to be quite flaky with regard to the USB device being hotplugged. On the other hand, the routed NAT + -o usb0 method is a lot more robust - it just starts working when the USB device is plugged in. I think I'll have to stick with the latter method (and just accept some excessive NATting) for now - this seems like it might be more of a Linux kernel / iptables issue than OpenWRT.

[jeff]

If you get curious, looking at the ubus events may give more insight into the hotplug “issues”.

If you don’t find the right page on the wiki, post again and I’ll dig it up when not on a phone browser.

[lleachii]

• @jeff could he do a script to reload the firewall on the USB Interface up?
• @wh2k9, this should only be a temporary workaround

[wh2k9]

I found this page, which suggests that you can do a script on ifup. The question is, do ifup events fire for individual interfaces that are part of a bridge?

[vgaetera]

I suggest to compare iptables dumps:

# When it's working
iptables-save > iptables-save.0
# When it's not working
iptables-save > iptables-save.1
# Compare dumps
diff iptables-save.*

[wh2k9]

I finally figured it out and now have this setup working reliably in bridge mode. The problem was actually in the sysctl settings for br_netfilter not being applied at startup (see http://ebtables.netfilter.org/documentation/bridge-nf.html). I just needed to put

net.bridge.bridge-nf-call-arptables = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

in /etc/sysctl.conf.

I also created a simple script /etc/hotplug.d/net/99-usb-mangle-ttl that will automatically apply and remove the iptables rule when usb0 is connected/disconnected from the system:

#!/bin/sh

if [ "$INTERFACE" = "usb0" ]; then
    if [ "$ACTION" = "add" ]; then
        iptables -t mangle -I FORWARD -m physdev --physdev-is-bridged --physdev-in eth1 --physdev-out usb0 -j TTL --ttl-set 65
    elif [ "$ACTION" = "remove" ]; then
        iptables -t mangle -D FORWARD -m physdev --physdev-is-bridged --physdev-in eth1 --physdev-out usb0 -j TTL --ttl-set 65
    fi
fi

It's not possible to use the higher-level iface hotplug system since it seems ifup/ifdown events are filtered for interfaces that are part of a bridge. In any case, it's probably not necessary to have a script in the first place since you should be able to stick it in the custom firewall rules section in the web GUI, but I think this is a more elegant approach.

Hopefully this saves some headaches for anyone else trying to do a similar setup.