Ubus calls not working properly for custom build (originally: WAN masquerade not forwarding traffic)

I'm trying to use OpenWRT has a simple router on my system. It has 5 interfaces (eth0 through eth4) and I'd like to configure eth0 as the WAN port, and eth1-4 as the LAN ports. On the router itself, I can access the internet, however traffic from LAN is not getting through. SNAT/masquerade doesn't seem to be working properly. If I create my own masquerade rule using iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE, everything seems to work normally. Configs below. Any ideas?

network config:

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config interface 'lan'
        list ifname 'eth1'
        list ifname 'eth2'
        list ifname 'eth3'
        list ifname 'eth4'
        option proto 'static'
        option type 'bridge'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'
        option metric '100'

config interface 'wan'
        option ifname eth0

config globals 'globals'
        option ula_prefix 'auto'

Firewall config:

config defaults
        option syn_flood '1'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'lan'
        list network 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'

config zone
        option name 'wan'
        list network 'wan'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'

config forwarding
        option src 'lan'
        option dest 'wan'

iptables-save output:

# Generated by iptables-save v1.8.4 on Fri Dec 22 23:00:03 2023
*nat
:PREROUTING ACCEPT [478405:197245537]
:INPUT ACCEPT [8564:1321355]
:OUTPUT ACCEPT [2108:149540]
:POSTROUTING ACCEPT [2348:168077]
:postrouting_lan_rule - [0:0]
:postrouting_rule - [0:0]
:postrouting_wan_rule - [0:0]
:prerouting_lan_rule - [0:0]
:prerouting_rule - [0:0]
:prerouting_wan_rule - [0:0]
:zone_lan_postrouting - [0:0]
:zone_lan_prerouting - [0:0]
:zone_wan_postrouting - [0:0]
:zone_wan_prerouting - [0:0]
[1039291:427710120] -A PREROUTING -m comment --comment "!fw3: Custom prerouting rule chain" -j prerouting_rule
[6711:483778] -A POSTROUTING -m comment --comment "!fw3: Custom postrouting rule chain" -j postrouting_rule
[0:0] -A zone_lan_postrouting -m comment --comment "!fw3: Custom lan postrouting rule chain" -j postrouting_lan_rule
[0:0] -A zone_lan_prerouting -m comment --comment "!fw3: Custom lan prerouting rule chain" -j prerouting_lan_rule
[0:0] -A zone_wan_postrouting -m comment --comment "!fw3: Custom wan postrouting rule chain" -j postrouting_wan_rule
[0:0] -A zone_wan_postrouting -m comment --comment "!fw3" -j MASQUERADE
[0:0] -A zone_wan_prerouting -m comment --comment "!fw3: Custom wan prerouting rule chain" -j prerouting_wan_rule
COMMIT
# Completed on Fri Dec 22 23:00:03 2023
# Generated by iptables-save v1.8.4 on Fri Dec 22 23:00:03 2023
*mangle
:PREROUTING ACCEPT [1148624:458996595]
:INPUT ACCEPT [94259:18687551]
:FORWARD ACCEPT [35431:15559536]
:OUTPUT ACCEPT [65201:7773625]
:POSTROUTING ACCEPT [100632:23333161]
COMMIT
# Completed on Fri Dec 22 23:00:03 2023
# Generated by iptables-save v1.8.4 on Fri Dec 22 23:00:03 2023
*filter
:INPUT ACCEPT [40443:6288514]
:FORWARD ACCEPT [7205:476445]
:OUTPUT ACCEPT [2773:190018]
:forwarding_lan_rule - [0:0]
:forwarding_rule - [0:0]
:forwarding_wan_rule - [0:0]
:input_lan_rule - [0:0]
:input_rule - [0:0]
:input_wan_rule - [0:0]
:output_lan_rule - [0:0]
:output_rule - [0:0]
:output_wan_rule - [0:0]
:reject - [0:0]
:syn_flood - [0:0]
:zone_lan_dest_ACCEPT - [0:0]
:zone_lan_forward - [0:0]
:zone_lan_input - [0:0]
:zone_lan_output - [0:0]
:zone_lan_src_ACCEPT - [0:0]
:zone_wan_dest_ACCEPT - [0:0]
:zone_wan_forward - [0:0]
:zone_wan_input - [0:0]
:zone_wan_output - [0:0]
:zone_wan_src_ACCEPT - [0:0]
[19913:1644097] -A INPUT -i lo -m comment --comment "!fw3" -j ACCEPT
[74346:17043454] -A INPUT -m comment --comment "!fw3: Custom input rule chain" -j input_rule
[33903:10754940] -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[0:0] -A INPUT -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m comment --comment "!fw3" -j syn_flood
[35431:15559536] -A FORWARD -m comment --comment "!fw3: Custom forwarding rule chain" -j forwarding_rule
[28226:15083091] -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[19913:1644097] -A OUTPUT -o lo -m comment --comment "!fw3" -j ACCEPT
[45288:6129528] -A OUTPUT -m comment --comment "!fw3: Custom output rule chain" -j output_rule
[42515:5939510] -A OUTPUT -m conntrack --ctstate RELATED,ESTABLISHED -m comment --comment "!fw3" -j ACCEPT
[0:0] -A reject -p tcp -m comment --comment "!fw3" -j REJECT --reject-with tcp-reset
[0:0] -A reject -m comment --comment "!fw3" -j REJECT --reject-with icmp-port-unreachable
[0:0] -A syn_flood -p tcp -m tcp --tcp-flags FIN,SYN,RST,ACK SYN -m limit --limit 25/sec --limit-burst 50 -m comment --comment "!fw3" -j RETURN
[0:0] -A syn_flood -m comment --comment "!fw3" -j DROP
[0:0] -A zone_lan_forward -m comment --comment "!fw3: Custom lan forwarding rule chain" -j forwarding_lan_rule
[0:0] -A zone_lan_forward -m comment --comment "!fw3: Zone lan to wan forwarding policy" -j zone_wan_dest_ACCEPT
[0:0] -A zone_lan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_lan_forward -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
[0:0] -A zone_lan_input -m comment --comment "!fw3: Custom lan input rule chain" -j input_lan_rule
[0:0] -A zone_lan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[0:0] -A zone_lan_input -m comment --comment "!fw3" -j zone_lan_src_ACCEPT
[0:0] -A zone_lan_output -m comment --comment "!fw3: Custom lan output rule chain" -j output_lan_rule
[0:0] -A zone_lan_output -m comment --comment "!fw3" -j zone_lan_dest_ACCEPT
[0:0] -A zone_wan_forward -m comment --comment "!fw3: Custom wan forwarding rule chain" -j forwarding_wan_rule
[0:0] -A zone_wan_forward -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port forwards" -j ACCEPT
[0:0] -A zone_wan_forward -m comment --comment "!fw3" -j zone_wan_dest_ACCEPT
[0:0] -A zone_wan_input -m comment --comment "!fw3: Custom wan input rule chain" -j input_wan_rule
[0:0] -A zone_wan_input -m conntrack --ctstate DNAT -m comment --comment "!fw3: Accept port redirections" -j ACCEPT
[0:0] -A zone_wan_input -m comment --comment "!fw3" -j zone_wan_src_ACCEPT
[0:0] -A zone_wan_output -m comment --comment "!fw3: Custom wan output rule chain" -j output_wan_rule
[0:0] -A zone_wan_output -m comment --comment "!fw3" -j zone_wan_dest_ACCEPT
COMMIT
# Completed on Fri Dec 22 23:00:03 2023

The masquerade rule doesn't seem to be getting any traffic.

There is no proto on the wan which means it won't have an IP address to masquerade from. Usually proto dhcp is used here to pull configuration from the ISP or other upstream network.

1 Like

-D is delete, Not Insert or append.

And there is no need to do that "manually". Put that rule into /etc/config/firewall .

What device and version of OpenWrt is this:

ubus call system board

Thank you all. I copied the wrong iptables rule into the first forum post, the one I added manually was done using -A.

Using option proto dhcp does help, but the DHCP script fails to add routes which breaks the path to the internet. I added -x to the DHCP script in /lib/netifd/dhcp.script to debug and I see that every call to ubus call network.interface notify_proto returns Command failed: Not found. Confirmed that this is the issue because using the default uDHCP script in /usr/share/udhcpc/default.script makes everything work correctly, but obviously this will break if netifd ever reloads the network configuration. Example:

+ ubus call network.interface notify_proto '{ "action": 0, "link-up": false, "keep": false, "interface": "" }'
Command failed: Not found
+ ubus call network.interface notify_proto '{ "action": 0, "link-up": true, "data": { "ntpserver": "192.168.4.1", "leasetime": 14400 }, "keep": false, "ipaddr": [ { "ipaddr": "192.168.4.252", "mask": "255.255.252.0" } ], "routes": [ { "target": "0.0.0.0", "netmask": "0", "gateway": "192.168.4.1", "source": "192.168.4.252" } ], "dns": [ "75.75.75.75", "75.75.76.76" ], "interface": "" }'
Command failed: Not found

Similar for calling system board.

# ubus call system board
Command failed: Not found

I've compiled OpenWRT using the Yocto layer here to run on a custom board: https://github.com/kraj/meta-openwrt so it's likely something is wrong with the way I'm compiling it or forgetting to start some services. Understand if that limits the support you can provide but any help is greatly appreciated!

This is not from the official OpenWrt project.

It appears you are using firmware that is not from the official OpenWrt project.

When using forks/offshoots/vendor-specific builds that are "based on OpenWrt", there may be many differences compared to the official versions (hosted by OpenWrt.org). Some of these customizations may fundamentally change the way that OpenWrt works. You might need help from people with specific/specialized knowledge about the firmware you are using, so it is possible that advice you get here may not be useful.

You may find that the best options are:

  1. Install an official version of OpenWrt, if your device is supported (see https://firmware-selector.openwrt.org).
  2. Ask for help from the maintainer(s) or user community of the specific firmware that you are using.
  3. Provide the source code for the firmware so that users on this forum can understand how your firmware works (OpenWrt forum users are volunteers, so somebody might look at the code if they have time and are interested in your issue).

If you believe that this specific issue is common to generic/official OpenWrt and/or the maintainers of your build have indicated as such, please feel free to clarify.

Thanks! I understand that. I'm actually using the firmware from git.openwrt.org. The repo I linked is just a set of build tools. Example netifd build: https://github.com/kraj/meta-openwrt/blob/master/recipes-core/netifd/netifd_git.bb#L6. The OpenWRT packages are all listed here: https://github.com/kraj/meta-openwrt/tree/master/recipes-core and I'm choosing to install netifd, firewall3, ubus and uci. My guess is I'm missing a package and that's why I see Command failed: not found.

Is there any reason you can't use the standard build envrionment from OpenWrt?

I have a bunch of existing code running alongside the OpenWRT processes that is already built using this build tool.

I'd recommend that you use an official OpenWrt build for your device to ensure that your configuration makes sense and is working as expected. If it doesn't work properly, then we can help you diagnose the potential issues since you'll be running pure OpenWrt (once changes are introduced relative to the official build tools and packages, all bets are off as to what impact those changes could have on the operation of the firmware).

ubus is a fundamental part of OpenWrt to pass configuration around. If ubus doesn't work you're going to have a lot of problems.

1 Like

That makes sense. It seems like ubus is working partially - I can make other calls using ubus call, it's just this notify_proto command that fails. I'm not sure how to debug that further.

Are you developing your own code, or just building/integrating code from others? If the latter, you should reach out to the maintainers of that code to find out what might be going wrong.

Developing my own code but using that build layer I linked as a starting point. I'm pretty new to OpenWRT so I'm not sure how to debug stuff like this.

I'll move this thread to the developers section where it may be seen by the most relevant audience... but the first thing that the devs are likely to say is to start with the default OpenWrt toolchain and build process and then build your code against that.

2 Likes