Although this fix is developed for inclusion in Mesh11sd Customer Premises Equipment (CPE), it could certainly be of general interest, particularly when another downstream router is deployed.
- see below for definitions of terms used
I recently ran into an issue where Mesh11sd CPE nodes (configured with portal_detect=3) were not appearing in the mesh11sd connect cli output list, even though they were fully working on the mesh backhaul.
Symptoms
-
The admin tool
mesh11sd connectdid not list CPE nodes. -
Stopping the firewall on the CPE node (service firewall stop) immediately made it visible (although blocked all access to users of the CPE node).
-
The CPE nodes themselves worked normally for clients and backhaul traffic.
Root Cause
On CPE nodes the mesh backhaul interface lives on br-wan (WAN zone). The default fw4 rules in chain input_wan do not accept incoming ICMPv6 traffic destined to the multicast address ff02::2. In addition, the Echo Replies sent back by the CPE were being dropped in chain accept_to_wan because conntrack marked them as ct state invalid.
Solution
On CPE nodes only, Mesh11sd now inserts two nft rules directly into the fw4 chains at the end of the service daemon loop.
Rule 1: Accept incoming multicast discovery pings to ff02::2
nft insert rule inet fw4 input_wan meta l4proto ipv6-icmp ip6 daddr ff02::2 counter accept comment "mesh11sd: Allow multicast discovery from portal"
Rule 2: Allow outgoing ICMPv6 on br-wan (needed for Echo Replies)
nft insert rule inet fw4 accept_to_wan meta nfproto ipv6 meta l4proto ipv6-icmp accept comment "mesh11sd: Allow ICMPv6 output on WAN (CPE discovery)"
These rules are inserted using nft insert (not UCI) because fw4’s UCI rule ordering cannot guarantee the rule appeared early enough in the chain.
In Mesh11sd, the logic is contained in a new function that is called at the end of the main daemon loop:
ensure_cpe_multicast_discovery_rule()
This only runs when portal_detect=3 (CPE mode) is set and the rules do not already exist in the nft ruleset.
Testing
Tested on a 20 node mesh, 5 of which were in a cable connected island mesh. A CPE node was placed in turn on mainland and island backhaul zones.
With these new rules in place, mesh11sd connect correctly lists the CPE node and remote access (ssh and scp) via mesh11sd connect <node_id > works normally.
Generic CPE Use
Without the Mesh11sd daemon or a mesh backhaul, use of an admin tool similar to mesh11sd connect can be enabled with these rules. They would be best applied via an fw4 shell script activated on a start of fw4.
Definitions of Terms Used
- CPE - Customer (or Client) Premises Equipment - fancy name for a router connected to an upstream network, which in turn has a routed Internet feed. Commonly used in community networks, large venues with shared Internet feed, WISP implementations etc. A CPE is a routed device with a wan connection upstream and lan/wi-fi for local users. A Mesh11sd CPE uses the mesh virtual interface for the wan connection (mesh backhaul).
ff02::2, the ipv6 multicast address for router discoveryportal- A mesh node with an upstream Internet feedportal_detect=3- the Mesh11sd config option to enable CPE mode on a node.- Mainland - A contiguous mesh backhaul zone that has a portal node with an upstream Internet feed
- Island - A contiguous mesh backhaul zone that is out of radio range of the Mainland zone, but has a cable connection to it.
Keywords
portal_detect=3, CPE mode, multicast-discovery, ff02::2, mesh11sd connect, ndp_scan, firewall, fw4, nft, input_wan, accept_to_wan, island-mesh, mainland-mesh