Simple (BATMAN) Mesh Network Setup

I've recently invested in a number of Carambola2-I modules (Carambola2-I initial setup) to attempt to set up a mesh network. As this is my first attempt, and I have no previous knowledge on the subject, I rely (mostly) on tutorials. At the moment I'm attempting to create a setup with two devices, but five more should be arriving shortly, and be added to the mesh thereafter.

I've managed to install the (at the time) latest version of OpenWrt on the devices (18.06.4) and BATMAN following the guide on the OpenWrt website (https://openwrt.org/docs/guide-user/network/wifi/mesh/batman?rev=1555021785). Using the batctl n command I have confirmed that the two nodes are able to see each other.

Now I'm down to (what I believe to be) the last step: routing (useful) traffic over the mesh network. If I connect to clients to one node, they both get assigned an IP address and are able to talk to (ping) each other. When I connect one client to one node, and the other to another, they both get assigned an IP address, but aren't able to communicate. I'm assuming I have to set up some kind of VLAN/bridge to route the traffic from the WiFi interface to the mesh, but have no idea how. I'm also figuring that, if one client connects to one node, and the other to another, they could technically get assigned the same IP address. I assume this means I should define one node as the DHCP server, and disable the functionality on all others. Should I set the nodes to be in a static range (and assign them all a static IP), and allow clients only a specific portion of IP addresses using one DHCP server?

My current configuration:
/etc/config/network

config wifi-device 'radio0'
        option type 'mac80211'
        option hwmode '11g'
        option path 'platform/ar933x_wmac'
        option htmode 'HT20'
        option disabled '0'
        option channel '1'
        option country 'US'
        option legacy_rates '1'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option encryption 'none'
        option ssid 'firebot-node-001'

config wifi-iface 'mesh0'
        option device 'radio0'
        option ifname 'mesh0'
        option network 'nwi_mesh0'
        option mode 'mesh'
        option mesh_fwding '0'
        option mesh_id 'firebot-mesh'
        option encryption 'none'

/etc/config/wireless

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd16:e8f1:3b01::/48'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config interface 'wwan'
        option proto 'dhcp'

config interface 'nwi_mesh0'
        option ifname 'mesh0'
        option proto 'batadv'
        option mesh 'bat0'
#       option routing_algo 'BATMAN_IV'
        option mtu '2304'

AFAIK (I don't use BATMAN myself) its configuration has changed significantly after 18.06.x (to the better), so I'd suggest to use 19.07.0-rc1 or master instead of 18.06.x for your endeavour.

Thanks for your advice. I ran into that while attempting to get this far, but due to my lack of experience with OpenWrt and the numerous topics I've read where people have managed to brick their devices, I'd rather stick with the firmware release listed on the device's page (https://openwrt.org/toh/8devices/carambola2). This may of course just be me being too careful.

That said, that still leaves the question of how to set it up.

You will need to cut back to a single DHCP server and then bridge each of the “LAN” (or whatever you’ve called them) segments with the batman interface on each router.

Moving to 19.07 or master is highly recommend for any advanced networking.

As per your and @slh's advice, I've installed OpenWrt 19.07.0-rc1 on the devices, and set up batman-adv following the new configuration standards. It took a bit of work, seeing as the tutorial isn't 100% accurate (batman-adv installs batctl-tiny during installation, so if you want to use batctl (like I do to list all neighbors) you have to install that before installing batman-adv), but I've now got the "same" setup as I had previously: both nodes have their own WiFi network, and both are able to see each other over the mesh network (confirmed using batctl n).

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fded:c663:b961::/48'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0'
        option proto 'static'
        option ipaddr '192.168.1.1'
        option netmask '255.255.255.0'
        option ip6assign '60'

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config interface 'bat0'
        option proto 'batadv'
        option routing_algo 'BATMAN_IV'
        option aggregated_ogms 1
        option ap_isolation 0
        option bonding 0
        option fragmentation 1
        #option gw_bandwidth '10000/2000'
        option gw_mode 'off'
        #option gw_sel_class 20
        option log_level 0
        option orig_interval 1000
        option bridge_loop_avoidance 1
        option distributed_arp_table 1
        option multicast_mode 1
        option network_coding 0
        option hop_penalty 30
        option isolation_mark '0x00000000/0x00000000'

config interface 'nwi_mesh0'
        option mtu '2304'
        option proto 'batadv_hardif'
        option master 'bat0'

/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/ar933x_wmac'
        option htmode 'HT20'
        option disabled '0'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'firebot-node-001'
        option encryption 'none'

config wifi-iface 'mesh0'
        option device 'radio0'
        option ifname 'mesh0'
        option network 'nwi_mesh0'
        option mode 'mesh'
        option mesh_fwding '0'
        option mesh_id 'firebot-mesh'
        option encryption 'none'

Now, like said before, I should now disable the DHCP server on one of the two nodes (and all nodes to be set up in the future). Can I just remove the 'wan' and 'wan6' interfaces to achieve this, or should I go about it differently?

Then I need to set up a bridge between the lan and mesh connection. This is what the user guide (https://openwrt.org/docs/guide-user/network/wifi/mesh/batman) suggests:

config interface 'vlan1111'
        option type 'bridge'
        option stp '1'
        option ifname 'eth1.1111 bat0.1111'
        option proto 'static'
        option ipaddr '192.168.11.11'
        option netmask '255.255.255.0'
        option delegate '0'

I'm assuming that most of it is right, but I have two questions: first of all, should I change eth1 to eth0 (because my lan interface is named eth0)? And secondly, what is the use/advantage/disadvantage of setting an IP address on the bridge? Shouldn't the DHCP server handle the IP addresses? And should I therefore remove the statements here?

Concepts here that would be helpful are that the mesh routes batman packets among the nodes. If you've got the batman nodes aware of each other, that part of the config should be working then. MTU might need some tweaking later on, but if you've got the nodes talking of each other, the next step is to route traffic.

The routing of traffic is done by adding the batman interface to the "right" bridge. Think of it as working like a wire that connects the bridges.

In the example, the device I used happened to use eth1 for its private subnet that was bridged. Your device seems to use eth0 for "lan", so that is what you would bridge. You could add the batman interface to your existing "lan" bridge directly, or pick a "convenient" VLAN so that you can easily extend to guest, IoT, ... , subnets later.

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0 bat0'
        [...]

or, in my preference, something like

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0 bat0.100'
        [...]

I prefer not to use VLAN 1 and 2, as they are often "special" in OpenWrt config. There is nothing "magic" about VLAN 1111. It was intended as an example number.

Thanks for your reply and investing your time in trying to help me, but I don't get how I should go about setting up the bridge. I noticed the lan interface in my config is already of type bridge. Should I just change ifname to 'eth0 bat0' (or 'eth0 bat0.100'), or should I add a new entry?

Also, how do I disable the DHCP server on the second node, and tell it to use the first one as a DHCP server? I'm assuming the first node should have a static IP address (192.168.1.1), and I should provide that to all the other ones, but I have no idea how.

Change the entry -- the "ifname" entry for a bridge is singular. See https://openwrt.org/docs/guide-user/base-system/basic-networking#options_valid_for_all_protocol_types for more details.

Disable DHCP on the other nodes, such as by disabling the dnsmasq service.

@jeff Thanks again for all your help. I installed the latest release once again (19.07.0), and using your answers in this topic and other topics I managed to get the mesh setup working.

For future reference, I disabled the DHCP and DHCPv6 servers using commands:

/etc/init.d/dnsmasq disable
/etc/init.d/dnsmasq stop
/etc/init.d/odhcpd disable
/etc/init.d/odhcpd stop

I then set the dns and gateway options of the lan interface to the IP address of the main node, and added bat0 to the iface:

/etc/config/network

config interface 'loopback'
        option ifname 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd78:d70e:087c::/48'

config interface 'lan'
        option type 'bridge'
        option ifname 'eth0 bat0'        # added bat0 here
        option proto 'static'
        option ipaddr '192.168.1.xxx'    # 192.168.1.1, .1.2, .1.3, ...
        option netmask '255.255.255.0'
        option ip6assign '60'
        option gateway '192.168.1.1'     # only on the secondary nodes
        option dns '192.168.1.1'         # only on the secondary nodes

config interface 'wan'
        option ifname 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option ifname 'eth1'
        option proto 'dhcpv6'

config interface 'wwan'
        option proto 'dhcp'

config interface 'bat0'
        option proto 'batadv'
        option routing_algo 'BATMAN_IV'
        option aggregated_ogms 1
        option ap_isolation 0
        option bonding 0
        option fragmentation 1
        #option gw_bandwidth '10000/2000'
        option gw_mode 'off'
        #option gw_sel_class 20
        option log_level 0
        option orig_interval 1000
        option bridge_loop_avoidance 1
        option distributed_arp_table 1
        option multicast_mode 1
        option network_coding 0
        option hop_penalty 30
        option isolation_mark '0x00000000/0x00000000'

config interface 'nwi_mesh0'
        option mtu '2304'
        option proto 'batadv_hardif'
        option master 'bat0'

As for the wireless config: it didn't need any alterations.
/etc/config/wireless

config wifi-device 'radio0'
        option type 'mac80211'
        option channel '11'
        option hwmode '11g'
        option path 'platform/ar933x_wmac'
        option htmode 'HT20'
        option disabled '0'

config wifi-iface 'default_radio0'
        option device 'radio0'
        option network 'lan'
        option mode 'ap'
        option ssid 'firebot-node-001'
        option encryption 'none'

config wifi-iface 'mesh0'
        option device 'radio0'
        option ifname 'mesh0'
        option mode 'mesh'
        option mesh_fwding '0'
        option mesh_id 'firebot-mesh'
        option encryption 'none'
        option network 'nwi_mesh0'

This topic was automatically closed 10 days after the last reply. New replies are no longer allowed.